Audit and clean up review notes: type consistency, naming, and lifecycle #246

Open
opened 2026-03-28 22:57:40 +00:00 by forgejo_admin · 4 comments
Contributor

Type

Bug

Lineage

Discovered during westside schedule feature scoping — review agents created notes with wrong slugs (Forgejo issue number instead of board item ID), wrong note_type (doc instead of review), blocking the check-board-advance hook.

Repo

forgejo_admin/pal-e-api (notes + skill-review-ticket)

Cross-repo touch: AC4 (update feedback_active_note_types) requires editing MEMORY.md in claude-custom, not this repo. Coordinate accordingly.

What Broke

Review notes are inconsistent and accumulating without cleanup:

  1. Slug naming: skill-review-ticket step 12 says review-{board_item_id}-YYYY-MM-DD but a small number of agents have used Forgejo issue numbers instead. The check-board-advance hook queries for review-{board_item_id}-*, so misnamed notes fail the gate silently. Note: audit shows the IDs used are generally in the board-item-ID range — this is not a systemic pattern, but the naming convention should still be reinforced.
  2. note_type: Skill says review but audit shows 44 of 301 review-tagged notes have the wrong type (42 doc + 2 null). The remaining 257 (85%) already have note_type=review. Fix scope is the 44, not 301.
  3. No lifecycle: Review notes accumulate forever. No convention for cleanup after tickets reach done.
  4. Active types list: review is not in feedback_active_note_types (sop, convention, doc, project-page, board). Either add it or decide reviews shouldn't be notes.

Repro Steps

  1. Run /review-ticket board-westside-basketball#629
  2. Review agent creates review-232-2026-03-28 (using Forgejo issue #232 instead of board item #629)
  3. Attempt to advance item #629 from todo to next_up
  4. check-board-advance hook blocks because it looks for review-629-*

Expected Behavior

  • Review notes use review-{board_item_id}-* slugs consistently
  • Hook finds them on first attempt
  • Old review notes get cleaned up when tickets reach done

Environment

  • pal-e-docs API: production
  • Hook: ~/.claude/hooks/check-board-advance.sh
  • Skill: skill-review-ticket note in pal-e-docs

Acceptance Criteria

  • All existing review notes audited: fix note_type to review for the 44 notes that need it (42 doc + 2 null out of 301 total)
  • Orphaned/duplicate review notes identified (notes where the board item is in done or no longer exists)
  • Convention note created: convention-review-note-lifecycle defining creation, naming, type, and cleanup rules
  • review added to active note types list in feedback_active_note_types in claude-custom MEMORY.md (or decision documented to remove review notes entirely)
  • skill-review-ticket step 12 verified — if the naming convention is clear, note WHY agents drift (possible: agents see existing misnamed notes and copy the pattern)
  • Discovered scope: template-review contains stale guidance ("Note type: doc (until review is added to NoteType enum)") — review IS already in the NoteType enum. Update the template to say note_type: review.
  • skill-review-ticket — the skill that guides review agents
  • check-board-advance.sh — hook in claude-custom that enforces the gate (code change is a separate ticket if needed)
  • feedback_active_note_types — memory noting active types (lives in claude-custom MEMORY.md, not pal-e-api)
  • convention-todo-lifecycle — related lifecycle convention
  • template-review — review note template (contains stale note_type guidance)
### Type Bug ### Lineage Discovered during westside schedule feature scoping — review agents created notes with wrong slugs (Forgejo issue number instead of board item ID), wrong note_type (`doc` instead of `review`), blocking the `check-board-advance` hook. ### Repo `forgejo_admin/pal-e-api` (notes + skill-review-ticket) **Cross-repo touch:** AC4 (update `feedback_active_note_types`) requires editing `MEMORY.md` in `claude-custom`, not this repo. Coordinate accordingly. ### What Broke Review notes are inconsistent and accumulating without cleanup: 1. **Slug naming:** `skill-review-ticket` step 12 says `review-{board_item_id}-YYYY-MM-DD` but a small number of agents have used Forgejo issue numbers instead. The `check-board-advance` hook queries for `review-{board_item_id}-*`, so misnamed notes fail the gate silently. Note: audit shows the IDs used are generally in the board-item-ID range — this is not a systemic pattern, but the naming convention should still be reinforced. 2. **note_type:** Skill says `review` but audit shows 44 of 301 review-tagged notes have the wrong type (42 `doc` + 2 `null`). The remaining 257 (85%) already have `note_type=review`. Fix scope is the 44, not 301. 3. **No lifecycle:** Review notes accumulate forever. No convention for cleanup after tickets reach `done`. 4. **Active types list:** `review` is not in `feedback_active_note_types` (sop, convention, doc, project-page, board). Either add it or decide reviews shouldn't be notes. ### Repro Steps 1. Run `/review-ticket board-westside-basketball#629` 2. Review agent creates `review-232-2026-03-28` (using Forgejo issue #232 instead of board item #629) 3. Attempt to advance item #629 from todo to next_up 4. `check-board-advance` hook blocks because it looks for `review-629-*` ### Expected Behavior - Review notes use `review-{board_item_id}-*` slugs consistently - Hook finds them on first attempt - Old review notes get cleaned up when tickets reach `done` ### Environment - pal-e-docs API: production - Hook: `~/.claude/hooks/check-board-advance.sh` - Skill: `skill-review-ticket` note in pal-e-docs ### Acceptance Criteria - [ ] All existing review notes audited: fix note_type to `review` for the 44 notes that need it (42 `doc` + 2 `null` out of 301 total) - [ ] Orphaned/duplicate review notes identified (notes where the board item is in `done` or no longer exists) - [ ] Convention note created: `convention-review-note-lifecycle` defining creation, naming, type, and cleanup rules - [ ] `review` added to active note types list in `feedback_active_note_types` in `claude-custom` MEMORY.md (or decision documented to remove review notes entirely) - [ ] `skill-review-ticket` step 12 verified — if the naming convention is clear, note WHY agents drift (possible: agents see existing misnamed notes and copy the pattern) - [ ] **Discovered scope:** `template-review` contains stale guidance ("Note type: doc (until review is added to NoteType enum)") — `review` IS already in the NoteType enum. Update the template to say `note_type: review`. ### Related - `skill-review-ticket` — the skill that guides review agents - `check-board-advance.sh` — hook in claude-custom that enforces the gate (code change is a separate ticket if needed) - `feedback_active_note_types` — memory noting active types (lives in `claude-custom` MEMORY.md, not pal-e-api) - `convention-todo-lifecycle` — related lifecycle convention - `template-review` — review note template (contains stale note_type guidance)
Author
Contributor

Review Note Audit -- Dottie Report

Summary

Metric Count
Total review notes found 21
Correct note_type (review) 18
Wrong note_type (doc instead of review) 3 (fixed)
Duplicate reviews (same board item, multiple reviews) 5 board items with review chains
Orphaned (board item in done or superseded) Needs board-level check per item (see below)

Review Notes Inventory

Slug Board Item Verdict note_type (before) note_type (after) Project
review-241-2026-03-28 #241 BLOCK review -- pal-e-docs
review-317-2026-03-24 #317 NEEDS_REFINEMENT review -- pal-e-docs
review-356-2026-03-25 #356 NEEDS_REFINEMENT review -- westside-basketball
review-356-2026-03-25-r2 #356 READY review -- westside-basketball
review-364-2026-03-27 #364 NEEDS_REFINEMENT review -- pal-e-agency
review-364-2026-03-27-v2 #364 READY review -- pal-e-agency
review-376-2026-03-25 #376 NEEDS_REFINEMENT review -- svelte-playground
review-376-2026-03-25-r2 #376 READY review -- svelte-playground
review-398-2026-03-26 #398 READY review -- pal-e-agency
review-416-2026-03-25 #416 NEEDS_REFINEMENT review -- westside-basketball
review-416-2026-03-26 #416 READY review -- westside-basketball
review-418-2026-03-25 #418 NEEDS_REFINEMENT review -- pal-e-agency
review-418-2026-03-25-r2 #418 NEEDS_REFINEMENT review -- pal-e-agency
review-418-2026-03-27 #418 NEEDS_REFINEMENT review -- pal-e-agency
review-424-2026-03-26 #424 READY review -- pal-e-docs
review-464-2026-03-27 #464 NEEDS_REFINEMENT review -- pal-e-platform
review-464-2026-03-28 #464 READY review -- pal-e-platform
review-479-2026-03-27 #479 NEEDS_REFINEMENT review -- pal-e-agency
review-181-2026-03-27 #181 NEEDS_REFINEMENT review -- pal-e-agency
review-595-2026-03-28 #595 NEEDS_REFINEMENT doc review (fixed) westside-ai-assistant
review-595-2026-03-28-r2 #595 READY review -- westside-ai-assistant
review-597-2026-03-28 #597 NEEDS_REFINEMENT doc review (fixed) westside-ai-assistant
review-597-2026-03-28-r2 #597 READY doc review (fixed) westside-ai-assistant

Fixes Applied

  1. Fixed note_type on 3 notes: review-595-2026-03-28, review-597-2026-03-28, review-597-2026-03-28-r2 changed from doc to review.
  2. Root cause: template-review naming convention section said note_type: doc (until review is added to NoteType enum) -- but review IS a valid NoteType now. Stale guidance caused the westside-ai-assistant session to use doc.

Duplicate Reviews (Review Chains)

These are expected -- they are the review-fix-re-review loop in action:

  • #356: 2 reviews (NEEDS_REFINEMENT -> READY)
  • #364: 2 reviews (NEEDS_REFINEMENT -> READY)
  • #376: 2 reviews (NEEDS_REFINEMENT -> READY)
  • #416: 2 reviews (NEEDS_REFINEMENT -> READY)
  • #418: 3 reviews (all NEEDS_REFINEMENT -- no final READY yet)
  • #464: 2 reviews (NEEDS_REFINEMENT -> READY)
  • #595: 2 reviews (NEEDS_REFINEMENT -> READY)
  • #597: 2 reviews (NEEDS_REFINEMENT -> READY)

Naming Convention Conflict Found

Three sources gave conflicting slug guidance:

  • skill-review-ticket step 12: review-{board_item_id}-{YYYY-MM-DD} (authoritative)
  • template-review: review-{issueNumber}-{YYYY-MM-DD} (stale)
  • note-conventions: review-{repo}-{pr-number} (describes PR reviews, different artifact)

Resolution: skill-review-ticket is authoritative. All existing review notes already follow the board-item-ID pattern. template-review naming convention section needs updating to match.

Convention Note Created

convention-review-note-lifecycle -- covers naming, note_type, tags, status, cleanup rules, and documents the source conflict resolution.

Remaining Work (for Betty Sue)

  1. Update template-review naming convention section to say review-{board_item_id} and note_type: review (currently says {issueNumber} and doc).
  2. Board item #418 has 3 NEEDS_REFINEMENT reviews with no READY -- the ticket refinement loop may be stuck.
  3. Orphan cleanup requires checking each board item's current column. Review notes for items in done older than 7 days are cleanup candidates per the new convention.
## Review Note Audit -- Dottie Report ### Summary | Metric | Count | |--------|-------| | Total review notes found | 21 | | Correct note_type (`review`) | 18 | | Wrong note_type (`doc` instead of `review`) | 3 (fixed) | | Duplicate reviews (same board item, multiple reviews) | 5 board items with review chains | | Orphaned (board item in `done` or superseded) | Needs board-level check per item (see below) | ### Review Notes Inventory | Slug | Board Item | Verdict | note_type (before) | note_type (after) | Project | |------|-----------|---------|--------------------|--------------------|---------| | `review-241-2026-03-28` | #241 | BLOCK | review | -- | pal-e-docs | | `review-317-2026-03-24` | #317 | NEEDS_REFINEMENT | review | -- | pal-e-docs | | `review-356-2026-03-25` | #356 | NEEDS_REFINEMENT | review | -- | westside-basketball | | `review-356-2026-03-25-r2` | #356 | READY | review | -- | westside-basketball | | `review-364-2026-03-27` | #364 | NEEDS_REFINEMENT | review | -- | pal-e-agency | | `review-364-2026-03-27-v2` | #364 | READY | review | -- | pal-e-agency | | `review-376-2026-03-25` | #376 | NEEDS_REFINEMENT | review | -- | svelte-playground | | `review-376-2026-03-25-r2` | #376 | READY | review | -- | svelte-playground | | `review-398-2026-03-26` | #398 | READY | review | -- | pal-e-agency | | `review-416-2026-03-25` | #416 | NEEDS_REFINEMENT | review | -- | westside-basketball | | `review-416-2026-03-26` | #416 | READY | review | -- | westside-basketball | | `review-418-2026-03-25` | #418 | NEEDS_REFINEMENT | review | -- | pal-e-agency | | `review-418-2026-03-25-r2` | #418 | NEEDS_REFINEMENT | review | -- | pal-e-agency | | `review-418-2026-03-27` | #418 | NEEDS_REFINEMENT | review | -- | pal-e-agency | | `review-424-2026-03-26` | #424 | READY | review | -- | pal-e-docs | | `review-464-2026-03-27` | #464 | NEEDS_REFINEMENT | review | -- | pal-e-platform | | `review-464-2026-03-28` | #464 | READY | review | -- | pal-e-platform | | `review-479-2026-03-27` | #479 | NEEDS_REFINEMENT | review | -- | pal-e-agency | | `review-181-2026-03-27` | #181 | NEEDS_REFINEMENT | review | -- | pal-e-agency | | `review-595-2026-03-28` | #595 | NEEDS_REFINEMENT | **doc** | **review** (fixed) | westside-ai-assistant | | `review-595-2026-03-28-r2` | #595 | READY | review | -- | westside-ai-assistant | | `review-597-2026-03-28` | #597 | NEEDS_REFINEMENT | **doc** | **review** (fixed) | westside-ai-assistant | | `review-597-2026-03-28-r2` | #597 | READY | **doc** | **review** (fixed) | westside-ai-assistant | ### Fixes Applied 1. **Fixed note_type on 3 notes:** `review-595-2026-03-28`, `review-597-2026-03-28`, `review-597-2026-03-28-r2` changed from `doc` to `review`. 2. **Root cause:** `template-review` naming convention section said `note_type: doc (until review is added to NoteType enum)` -- but `review` IS a valid NoteType now. Stale guidance caused the westside-ai-assistant session to use `doc`. ### Duplicate Reviews (Review Chains) These are expected -- they are the review-fix-re-review loop in action: - **#356**: 2 reviews (NEEDS_REFINEMENT -> READY) - **#364**: 2 reviews (NEEDS_REFINEMENT -> READY) - **#376**: 2 reviews (NEEDS_REFINEMENT -> READY) - **#416**: 2 reviews (NEEDS_REFINEMENT -> READY) - **#418**: 3 reviews (all NEEDS_REFINEMENT -- no final READY yet) - **#464**: 2 reviews (NEEDS_REFINEMENT -> READY) - **#595**: 2 reviews (NEEDS_REFINEMENT -> READY) - **#597**: 2 reviews (NEEDS_REFINEMENT -> READY) ### Naming Convention Conflict Found Three sources gave conflicting slug guidance: - `skill-review-ticket` step 12: `review-{board_item_id}-{YYYY-MM-DD}` (authoritative) - `template-review`: `review-{issueNumber}-{YYYY-MM-DD}` (stale) - `note-conventions`: `review-{repo}-{pr-number}` (describes PR reviews, different artifact) **Resolution:** `skill-review-ticket` is authoritative. All existing review notes already follow the board-item-ID pattern. `template-review` naming convention section needs updating to match. ### Convention Note Created `convention-review-note-lifecycle` -- covers naming, note_type, tags, status, cleanup rules, and documents the source conflict resolution. ### Remaining Work (for Betty Sue) 1. **Update `template-review` naming convention section** to say `review-{board_item_id}` and `note_type: review` (currently says `{issueNumber}` and `doc`). 2. **Board item #418 has 3 NEEDS_REFINEMENT reviews with no READY** -- the ticket refinement loop may be stuck. 3. **Orphan cleanup** requires checking each board item's current column. Review notes for items in `done` older than 7 days are cleanup candidates per the new convention.
Author
Contributor

Scope Review: NEEDS_REFINEMENT

Review note: review-634-2026-03-28
Issue body contains inaccurate data claims that should be corrected before execution.

  • [BODY] What Broke #2 says "most notes were created as doc or null" — actual data: 257/301 (85%) already have correct note_type=review. Only 44 need fixing (42 doc + 2 null).
  • [BODY] What Broke #1 overstates slug naming problem — IDs 55-653 ARE board item IDs, not Forgejo issue numbers. The repro example may be valid but the claim of systemic misnaming is not supported by data.
  • [BODY] Add cross-repo note: AC4 touches claude-custom MEMORY.md, not pal-e-api.
  • [BODY] Discovered: template-review Naming Convention section says Note type: doc (until review is added to NoteType enum) but review IS already in the enum. Should be fixed as part of this work or as discovered scope.
## Scope Review: NEEDS_REFINEMENT Review note: `review-634-2026-03-28` Issue body contains inaccurate data claims that should be corrected before execution. - **[BODY]** What Broke #2 says "most notes were created as doc or null" — actual data: 257/301 (85%) already have correct `note_type=review`. Only 44 need fixing (42 doc + 2 null). - **[BODY]** What Broke #1 overstates slug naming problem — IDs 55-653 ARE board item IDs, not Forgejo issue numbers. The repro example may be valid but the claim of systemic misnaming is not supported by data. - **[BODY]** Add cross-repo note: AC4 touches `claude-custom` MEMORY.md, not pal-e-api. - **[BODY]** Discovered: `template-review` Naming Convention section says `Note type: doc (until review is added to NoteType enum)` but `review` IS already in the enum. Should be fixed as part of this work or as discovered scope.
Author
Contributor

Scope refinement applied (Dottie, 2026-03-28)

Source: review-634-2026-03-28 (NEEDS_REFINEMENT verdict, board-pal-e-agency item 634)

4 fixes applied to issue body:

  1. Corrected data claims (What Broke #2, AC1): Original said "most notes were created as doc or null" and "50+ notes tagged review exist with mixed types." Audit shows 257/301 (85%) already have note_type=review. Only 44 need fixing (42 doc + 2 null). AC1 updated to reflect the actual fix scope.

  2. Corrected slug naming claim (What Broke #1): Original said "agents frequently use Forgejo issue numbers instead of board item IDs." Audit shows the IDs are generally in the board-item-ID range — not a systemic pattern. Softened the language and noted the convention should still be reinforced.

  3. Documented cross-repo touch (AC4): feedback_active_note_types lives in claude-custom MEMORY.md, not pal-e-api. Added explicit callout in Repo section and updated AC4 + Related section to note the cross-repo dependency.

  4. Added discovered scope — template-review stale guidance (new AC6): template-review says "Note type: doc (until review is added to NoteType enum)" but review IS already in the NoteType enum. Added as a new checklist item in Acceptance Criteria and added template-review to Related section.

## Scope refinement applied (Dottie, 2026-03-28) Source: `review-634-2026-03-28` (NEEDS_REFINEMENT verdict, board-pal-e-agency item 634) ### 4 fixes applied to issue body: 1. **Corrected data claims (What Broke #2, AC1):** Original said "most notes were created as `doc` or `null`" and "50+ notes tagged `review` exist with mixed types." Audit shows 257/301 (85%) already have `note_type=review`. Only 44 need fixing (42 `doc` + 2 `null`). AC1 updated to reflect the actual fix scope. 2. **Corrected slug naming claim (What Broke #1):** Original said "agents frequently use Forgejo issue numbers instead of board item IDs." Audit shows the IDs are generally in the board-item-ID range — not a systemic pattern. Softened the language and noted the convention should still be reinforced. 3. **Documented cross-repo touch (AC4):** `feedback_active_note_types` lives in `claude-custom` MEMORY.md, not pal-e-api. Added explicit callout in Repo section and updated AC4 + Related section to note the cross-repo dependency. 4. **Added discovered scope — template-review stale guidance (new AC6):** `template-review` says "Note type: doc (until review is added to NoteType enum)" but `review` IS already in the NoteType enum. Added as a new checklist item in Acceptance Criteria and added `template-review` to Related section.
Author
Contributor

Scope Review: NEEDS_REFINEMENT

Review note: review-634-2026-03-29

Well-scoped bug with complete template and full traceability, but exceeds single-agent execution capacity.

  • AC2 ambiguity: "identified" is not a verifiable end state -- clarify deliverable (list? delete? archive?)
  • AC5 untestable: "Note WHY agents drift" is analysis, not a verifiable criterion -- split from the testable part (verify step 12 is correct)
  • Undocumented dependency: Board item #478 (note type system audit spike) is in_progress and may affect convention design (AC3) and active types list (AC4)
  • DECOMPOSE: 6 AC across 3 repos (pal-e-api, pal-e-docs notes, claude-custom) exceeds the three-thing limit -- recommend decomposition via template-board into 3-4 sub-tickets
## Scope Review: NEEDS_REFINEMENT Review note: `review-634-2026-03-29` Well-scoped bug with complete template and full traceability, but exceeds single-agent execution capacity. - **AC2 ambiguity:** "identified" is not a verifiable end state -- clarify deliverable (list? delete? archive?) - **AC5 untestable:** "Note WHY agents drift" is analysis, not a verifiable criterion -- split from the testable part (verify step 12 is correct) - **Undocumented dependency:** Board item #478 (note type system audit spike) is in_progress and may affect convention design (AC3) and active types list (AC4) - **DECOMPOSE:** 6 AC across 3 repos (pal-e-api, pal-e-docs notes, claude-custom) exceeds the three-thing limit -- recommend decomposition via template-board into 3-4 sub-tickets
Commenting is not possible because the repository is archived.
No milestone
No project
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ldraney/pal-e-api#246
No description provided.