summaryrefslogtreecommitdiff
path: root/project_claudomator_arbitration_fail_open_incident.md
diff options
context:
space:
mode:
authorClaude Code <claude@terst.org>2026-08-15 18:06:00 +0000
committerClaude Code <claude@terst.org>2026-08-15 18:06:00 +0000
commita6d58e009efcc06e7b5221d75ebed36533786dce (patch)
treecfeded4c1bb8e61a4c117aced0b42e7d4aa0e15a /project_claudomator_arbitration_fail_open_incident.md
parenteb0932bddad0daa47c36cd20599944c3dc45482a (diff)
Merge titanium's diverged memory tree into the shared historyHEADmain
Titanium and tungsten had grown two separate, unreconciled memory sets for the same -workspace scope. Ports over 18 titanium-only entries (12 claudomator files, the fork-scope-creep lesson, the doot Google Tasks OAuth limitation, the default-subagent-driven-execution preference, scout's Facebook priority, Hawaii GIS endpoints), merges two true duplicates into their existing tungsten files rather than keeping parallel copies (the migration-rename incident -- same 022->023 rename, same commit, titanium's version added the confirmed-safe-to-patch-prod-directly carve-out tungsten's didn't have; the APK build OOM note, folded into project_doot_widget_build.md as a RAM-constrained-host fallback), and marks doot_future_task_scheduling_ideas.md as historical now that all four of its backlog items (budgets, labels/projects, buckets, chains) are built.
Diffstat (limited to 'project_claudomator_arbitration_fail_open_incident.md')
-rw-r--r--project_claudomator_arbitration_fail_open_incident.md18
1 files changed, 18 insertions, 0 deletions
diff --git a/project_claudomator_arbitration_fail_open_incident.md b/project_claudomator_arbitration_fail_open_incident.md
new file mode 100644
index 0000000..475c4ee
--- /dev/null
+++ b/project_claudomator_arbitration_fail_open_incident.md
@@ -0,0 +1,18 @@
+---
+name: claudomator-arbitration-fail-open-incident
+description: FIXED 2026-07-11 (commits cc1aa8d/204fe6c) — a live story shipped factually-wrong work because arbitration never called report_verdict and the system defaulted to silent approval
+metadata:
+ node_type: memory
+ type: project
+ originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390
+---
+
+**Status: fixed**, commits `cc1aa8d`/`204fe6c` on `main`, deployed 2026-07-11. `StoryOrchestrator.finalizeArbitration` is now fail-closed: approval requires an explicit `report_verdict(approved: true, ...)` call; an explicit rejection OR no verdict reported at all both reject the node (previously, no verdict silently defaulted to approval). `SeedRoleConfigs` now also seeds `planner` (the arbitration role) with a system prompt that explicitly mandates calling `report_verdict` before finishing — the role previously had no system prompt at all, only `builder` did (piece 5's original scoping).
+
+**What happened (real incident, first live run of the finished recursive-arbitrated-review design):** the very first story run through the newly-deployed design (a "diagram claudomator's workflow" story, self-referential dogfooding) had `evaluator_correctness` correctly flag 3 factually-wrong data-flow claims in the produced diagram — but the arbitration task that read all 4 evaluators' findings never called `report_verdict` at all (no system prompt told it the tool existed or was mandatory). The old fail-open default treated "no verdict" the same as "approved," so the story moved to `REVIEW_READY` with broken work, and nothing in the system detected this — the user caught it by reading the evaluator's own verdict text and asking "shouldn't this eval have failed?"
+
+**Why this matters beyond the one bug:** this is the second gap traced to piece 5's deliberate choice to seed only `builder`'s system prompt, not the other 5 roles (`evaluator_*`, `planner`, `retro`) — see [[claudomator-task-board-accept-workflow]] session context. The first gap (no `role_configs` = no `Scheduler` retry safety net on transient failure) was found earlier the same session; this one (an unguided arbitration agent silently no-ops the entire review gate) is more serious, since it defeats the whole point of "arbitrated review" as quality control, not just availability.
+
+**How to apply:** If seeding role prompts for the other 4 roles (`evaluator_quality/security/correctness/performance`, `retro`) comes up, check whether any of them have a similarly load-bearing "must call this specific tool" requirement with no explicit instruction to do so — `retro`'s `propose_role_config`/`report_summary` calls are a plausible candidate to check.
+
+**Also surfaced during cleanup of this incident:** a rejected root task that had *already* been incorrectly approved (`READY → COMPLETED`) before the fix landed cannot be un-approved — `COMPLETED` is a terminal state in `task.ValidTransition`. Manually resetting the *story* back to `NEEDS_FIX` after that point is not enough by itself: `ensureFixAttempt`'s idempotency check looks for an existing `builder`-role dependent of the (now-permanently-`COMPLETED`) root and, once that fix-attempt itself later dies (e.g. an infra failure exhausting its retry ladder), deleting the dead fix-attempt task alone does *not* re-trigger `ensureFixAttempt` — the story's `status` has already moved to `IN_PROGRESS` and stays there, `task.CurrentAttempt` resolves back to the original `COMPLETED` root, and `processStory`'s `root.State == StateCompleted` short-circuit means nothing further ever happens for that story. The fix is to re-set `story.status = "NEEDS_FIX"` again after deleting the dead fix-attempt (not just delete the task) — `ensureFixAttempt` doesn't care that the anchor root is `COMPLETED`, it just needs the story status to actually be `NEEDS_FIX` to fire at all.