diff options
41 files changed, 979 insertions, 0 deletions
@@ -1,3 +1,45 @@ +# Memory Index + - [Workspace layout](workspace-layout.md) — what each repo under /workspace is (listen, feedreader, podkit, devtools, ...) - [Listen handover 2026-09-21](listen-handover-2026-09-21.md) — queue-advance & notif skip-forward unverified by Peter; Phase 2 decisions pending - [Fedora Android build setup](fedora-android-build-setup.md) — JDK21 at ~/.jdks, SDK at ~/Android/Sdk, fixes for stale app/build/ +- [Doot production deployment](project_doot_prod.md) — deploy host is tungsten; verified real paths/service name/deploy script as of 2026-07-17's first real deploy (prior versions were guessed from a resembling sandbox and wrong) +- [Doot widget APK build & publish](project_doot_widget_build.md) — bootstrap Gradle/Android SDK in sandbox, build debug (not release) APK, publish to /site/static.terst.org/public/files/doot-widget.apk +- [Always pull before building](feedback_pull_before_build.md) — git fetch/ff to latest before any build/publish, unprompted, every time — no exceptions +- [Never renumber migrations in merges](feedback_migration_renumbering.md) — give new migrations the next free number, never rename an existing one; doot tracks by filename and a rename broke a live deploy +- [Deploy classifier handling](feedback_deploy_classifier.md) — how to work with the auto-mode gate on deploy-shaped actions without trying to route around it; this user's single-user-infra attitude toward deploy caution +- [Doot "marine" project code map](reference_doot_marine_codemap.md) — "marine project" = doot task chain, not a repo; project_id/chain_id, DB path, Go files/routes +- [Marine Cert Prep context](project_marine_cert_prep.md) — NMEA MEI certification study plan, what the 16-step chain covers +- [Feedreader (FeeDreader, was "Feeds") app](project_feedreader.md) — greenfield Android RSS reader, org.terst.feedreader, hands-on-testing-rounds workflow +- [Feedreader emulator on tungsten](project_feedreader_emulator.md) — feedreader_test AVD for direct testing/debugging instead of phone-only feedback +- [Feedreader APK naming](feedback_feedreader_apk_naming.md) — send debug builds as feeds-debug.apk, not the default app-debug.apk +- [Feedreader: no implicit mark-read](feedback_feedreader_no_implicit_read.md) — only opening an article marks it read, never scrolling; don't import other apps' UX defaults on ambiguous asks +- [Don't re-enable after told off](feedback_dont_reenable_after_told_off.md) — when told to leave something off, verify OFF is respected; don't flip it on yourself to "keep testing" +- [Keep live verification lightweight](feedback_lightweight_verification.md) — don't loop screenshot-scroll cycles to visually confirm what's already proven at DB/HTML level +- [Widget color theming iteration](feedback_widget_color_theming.md) — structure vs color complaints are separate, no manual alpha over theme colors (incl. grid lines), 4-way theme enum, no native shadow API (use ShadowedText) +- [Google Tasks due-date UTC quirk](reference_google_tasks_due_date_utc_quirk.md) — API's date-only due field is always midnight-UTC serialized; don't `.In(displayTZ)` it, re-anchor Y/M/D instead +- [IsAllDay midnight bug](project_doot_isallday_midnight_bug.md) — FIXED 2026-08-10 (9a3ece4): dated tasks landed in widget's undated pool (shown as "today"); original fix plan would've broken web rendering, actual fix used a new Undated field instead +- [Verify before asserting root cause](feedback_verify_before_asserting_root_cause.md) — check server logs/live curl before proposing a cause for a prod bug report; don't ship a "fix" before confirming it's THE bug, not just A bug +- [Commit as default](feedback_commit_as_default.md) — on doot, commit finished/verified work as part of completing it; don't wait to be asked each time +- [Fix the entire bug class](feedback_fix_entire_bug_class.md) — patch the instance AND audit for/prevent the whole class (structural test/lint), like the context_audit_test.go example +- [Investigate the actual claim](feedback_investigate_the_actual_claim.md) — check the user's lead in their own words; don't paraphrase it into an easier claim and reject that instead (dex "packing" ≠ "bloat") +- [Doot Android roadmap](project_doot_android_roadmap.md) — glanceable-test seam confirmed; offline support is a real requirement, not hypothetical — build via web-app service worker, not native rewrite +- [Architecture review scope](feedback_architecture_review_scope.md) — structural analysis only, don't frame around incidents unless a post-mortem was actually requested +- [Doot future task scheduling ideas](doot_future_task_scheduling_ideas.md) — design-origin doc for chains/buckets/labels/projects (all four now built); rationale, not a backlog +- [Doot Google Tasks OAuth limitation](project_doot_google_tasks_oauth_limitation.md) — service-account auth can never see the user's real task lists (unlike Calendar); needs real 3-legged OAuth; user deferred fixing as of 2026-07-14 +- [Fork scope creep risk](feedback_fork_scope_creep_risk.md) — a narrowly-scoped fork inherited full context and did unauthorized work anyway; name every forbidden action explicitly, or use a fresh subagent instead +- [Default subagent-driven execution](feedback_default_subagent_driven_execution.md) — on doot, when a plan's execution-choice prompt appears, default to Subagent-Driven without waiting for the user to pick +- [Facebook priority in scout](feedback_facebook_priority.md) — Facebook is the priority source in scout, never de-emphasize or suggest dropping it +- [Hawaii GIS data sources](reference_hawaii_gis_sources.md) — verified ArcGIS REST endpoints for geodata.hawaii.gov and gis.hawaiicounty.gov; parcels, hazards, permits, sewer, traffic, zoning +- [Claudomator chatbot MCP](project_claudomator_chatbot_mcp.md) — remote-control MCP endpoint setup + settings.json-isn't-for-mcpServers gotcha +- [Claudomator sequential orchestration gap](project_claudomator_sequential_orchestration_gap.md) — retro: needs a task-chain orchestration mode distinct from StoryOrchestrator; favor code-enforced checks over prompt reminders +- [Claudomator no watered-down architecture](feedback_claudomator_no_watered_down_architecture.md) — recursive verification is mandatory at every level; don't frame forced answers as open design questions +- [Claudomator workspace disk leak](project_claudomator_workspace_disk_leak.md) — FIXED (06c9730); live server needs redeploy to pick it up +- [Claudomator task board accept workflow](feedback_claudomator_task_board_accept_workflow.md) — SDD loop never calls accept_task after review; call it right after APPROVE, not as a later cleanup pass +- [Claudomator CLAUDE.md staleness](project_claudomator_claude_md_staleness.md) — FIXED (6214aa8); Story orchestrator section now describes the finished design +- [Claudomator workflow diagram TODO](project_claudomator_workflow_diagram_todo.md) — captured as claudomator story 11bd1cff (BACKLOG), not yet built; needs live server redeploy first +- [Claudomator stories NOT NULL bug](project_claudomator_stories_not_null_bug.md) — POST /api/stories requires explicit validation/deploy_config despite "nullable" docs +- [Claudomator arbitration fail-open incident](project_claudomator_arbitration_fail_open_incident.md) — FIXED (cc1aa8d/204fe6c); live run shipped factually-wrong work via silent no-verdict approval; now fail-closed + planner seeded +- [Claudomator builder prompt decomposition bias](project_claudomator_builder_prompt_decomposition_bias.md) — retro hypothesis: bias builder's prompt toward decomposing more readily; not yet implemented +- [Claudomator Tasks tab tree view TODO](project_claudomator_tasks_tab_tree_view_todo.md) — replace flat Tasks tab with a live tree view rooted at stories; not yet implemented +- [Claudomator distributed execution idea](project_claudomator_distributed_execution_idea.md) — reuse NativeRunner per remote machine + escalation-ladder round-robin; check agentloop tool parity first diff --git a/doot_future_task_scheduling_ideas.md b/doot_future_task_scheduling_ideas.md new file mode 100644 index 0000000..bbb2e58 --- /dev/null +++ b/doot_future_task_scheduling_ideas.md @@ -0,0 +1,45 @@ +--- +name: doot-future-task-scheduling-ideas +description: "Design-origin doc for doot's chains/buckets/labels/projects features (all four now built, as of this session) — kept for the rationale behind current shape, not as a backlog" +metadata: + node_type: memory + type: project + originSessionId: 61c70ef2-1b1b-456c-9cbd-ad1f76ea2401 + modified: 2026-08-15T18:04:22.781Z +--- + +**Superseded 2026-08-14: all four ideas below are now built** ([[reference_doot_marine_codemap]], [[project_doot_isallday_midnight_bug]] and this session's work used chains/buckets/projects/labels directly against live doot). Originally captured 2026-07-15 on titanium as backlog "thoughts for the future," refined through a 24-question interview the same day, and had no spec yet at time of writing. Keeping this file for the *rationale* behind why chains are strict WIP-1, why buckets auto-select by staleness+priority, why labels and projects are two separate concepts, etc. — treat the content below as "why it works this way," not as a to-do list. + +## 1. Task budgets / availability times +- Availability is **both**: a manual recurring weekly pattern (e.g. "2hr weekday evenings, 4hr Saturday") as a baseline, further reduced by actual Google Calendar events. +- Budget is **both**: per-task duration estimates that roll up into and draw down a weekly/period pool. +- Behavior: **just surface fit/overflow** — no auto-scheduling into slots; user still picks what to work on. +- Scope: **opt-in per project/label** (depends on item 2 existing first) — not a global always-on constraint. +- Overflow handling: **just flag it** (a warning/indicator), no auto-reprioritization or auto-push. +- Estimate source: **user-entered, plus learned from history** (infer typical duration from past completions of similar tasks — needs completion-time tracking that doesn't exist yet). + +## 2. Task labels/projects +- **Two separate concepts**: one project (hierarchy/grouping, exactly one per task) + many cross-cutting labels (tags). +- Data model: reuse the `Task.Labels []string` field for tags; a separate lightweight Projects concept alongside it. +- Color coding: **yes**, user-assignable colors per label/project, matching the existing `sourceColor()` convention. +- Widget visibility: widget shows a color mark on the time grid; full label/project display and editing happens in the task-detail popup. +- Recurrence interaction: project/labels are **series-level metadata** — every new recurring occurrence inherits them automatically. +- Cardinality: **exactly one project per task**; labels handle the many-to-many cross-cutting need. + +## 3. Recurring maintenance buckets (pick N from a list every M days/weeks) +- Architecture: a new recurrence variant layered onto the existing recurrence system, not a parallel/separate feature. +- Selection mechanism: **system auto-selects N items** each cycle — not a user-chosen pool. +- Deferral: deferring a selected item **puts it back into the pool**, eligible again next cycle (not pushed further out specifically). +- Selection algorithm: **weighted/scored** — combining priority, staleness, and label/availability match. +- Item timing: **no fixed due dates on individual items** — the M-day/week cycle length is the only timing concept. +- Bucket membership: **exactly one bucket per item** (not multi-bucket). + +## 4. Linear task chains with WIP limits +- WIP limit: **exactly 1** active task per chain at a time (strict, not configurable). +- Structure: **strictly linear**, fixed order, no branching or reordering. +- Visibility of future (locked) tasks: **hidden from timelines**, but **visible in the tasks list**. +- Looping: **always terminates** — no loop-back-to-start option. +- Project tie-in: **yes, one project per chain** — a chain is effectively a project with strict sequential unlocking. +- Abandonment: **yes**, a chain can be paused/abandoned partway through without completing every task. + +**How to apply:** if a future feature request sounds like it's asking to change one of these four's fundamental shape (e.g. "let buckets have more than 1 item active," "let chains branch"), that's a deliberate departure from this original design, not a bug — confirm intent before assuming it's wrong. diff --git a/feedback_architecture_review_scope.md b/feedback_architecture_review_scope.md new file mode 100644 index 0000000..40f36b3 --- /dev/null +++ b/feedback_architecture_review_scope.md @@ -0,0 +1,14 @@ +--- +name: feedback-architecture-review-scope +description: "When asked for an architectural review, keep it structural — don't lean on incident narrative as evidence or hook" +metadata: + node_type: memory + type: feedback + originSessionId: 2607e018-d4ad-4ec6-bb68-5f1d0f9a3906 +--- + +When the user asks for an "architectural review," give structural analysis — how components relate, what the design trades off — not an incident post-mortem. Corrected 2026-08-07: a doot service-gateway review leaned on that day's outage (and an earlier 3-day incident) as the framing device and evidence for a "blast radius" argument; the user said "I'm not interested in an incident review, I believe I asked for an architectural review." + +**Why:** Incidents are useful *inputs* the reviewer can reason from privately, but citing them as the review's evidence/narrative conflates "this specific bug happened" with "this is a structural property of the design" — and the user reads the two as different documents entirely. + +**How to apply:** State structural costs/tradeoffs (e.g. "an in-process gateway means one process's failures take independent services down with it") as properties of the design, not as "here's what broke and when." If a past incident is genuinely relevant, it's fine to have privately informed the analysis — just don't put it in the document unless asked for a post-mortem specifically. diff --git a/feedback_claudomator_no_watered_down_architecture.md b/feedback_claudomator_no_watered_down_architecture.md new file mode 100644 index 0000000..f7ec74d --- /dev/null +++ b/feedback_claudomator_no_watered_down_architecture.md @@ -0,0 +1,20 @@ +--- +name: claudomator-no-watered-down-architecture +description: "User strongly rejects \"simplified\" or \"optional\" framings of claudomator's recursive-arbitrated-review architecture — verification must be real and recursive at every level, no exceptions offered as a design choice." +metadata: + node_type: memory + type: feedback + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +When working on claudomator's recursive-arbitrated-review design (docs/superpowers/specs/2026-07-09-recursive-arbitrated-review-design.md), do not propose scoped-down, "simpler," or "optional" variants of the recursion as if they were legitimate design forks — even when framed neutrally as a question for the user to decide. + +Two concrete incidents this session: +1. I proposed an "outer pipeline only, subtasks decompose freely" alternative to avoid touching core executor machinery. User's response (verbatim): "oh my God stop whining about existing code, paradigms and architecture. quit proposing this half baked shit... simple recursive model... because the turtles are simple things (built by simple parrots), they require supervision at all levels." +2. I later framed "should a roll-up wait for its children's arbitrated-review verdicts, not just raw task-completion state, before proceeding?" as an open question needing the user's decision. User's response: this has exactly one correct answer given the architecture's own premise — a roll-up cannot be verified until its subtasks are actually verified (arbitrated), not merely COMPLETED. Presenting it as a live fork read as evasion. + +**Why:** The user's mental model is "turtles all the way down" — one uniform recursive mechanism, applied identically at every depth, with real supervision (arbitrated review) at every level. Anything that special-cases the root, skips a level, or treats verification as optional at some depth violates the premise by construction, not by omission — it isn't a judgment call, it's a bug. + +**How to apply:** Before presenting an "open question" about this architecture to the user, check whether the answer is actually forced by principles already established (recursion is uniform; verification precedes completion at every level). If it's forced, just say so and implement it — don't manufacture a decision point. Reserve real questions for things that are genuinely underdetermined by the stated architecture (e.g., a concrete mechanism choice with no principled answer, not a scope-reduction in disguise). + +Related: [[claudomator-sequential-orchestration-gap]] diff --git a/feedback_claudomator_task_board_accept_workflow.md b/feedback_claudomator_task_board_accept_workflow.md new file mode 100644 index 0000000..0d2f0c4 --- /dev/null +++ b/feedback_claudomator_task_board_accept_workflow.md @@ -0,0 +1,14 @@ +--- +name: claudomator-task-board-accept-workflow +description: "SDD dispatch workflow never calls accept_task after a reviewed/merged claudomator task — board fills up with READY tasks that look unfinished even though they're done" +metadata: + node_type: memory + type: feedback + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +This session's subagent-driven-development loop (dispatch a claudomator task → implementer commits+pushes → review the diff via a separate local Agent subagent → update the progress ledger) never calls `mcp__claudomator__accept_task` on the claudomator task itself once review comes back APPROVE. The claudomator task board's own review/accept mechanism and this session's out-of-band local-subagent review process were two disconnected tracks — so every successfully-reviewed-and-merged piece this session (pieces 1 through 4b-3, all their retries) sat at READY on the task board indefinitely, looking like unactioned work even though it was actually done. User flagged this directly: "the task board is filling up with artifacts from your runs — we need to clean those up and also tighten the workflow so they're accurate end to end." + +**Why:** The local-subagent review step (dispatching a fresh Agent to read a git diff and verdict APPROVE/REJECT) is a code-quality gate this session invented on top of claudomator's own task lifecycle; nothing in that invented process closes the loop back to claudomator's actual `READY → COMPLETED` state transition. + +**How to apply:** Once a local review subagent returns APPROVE for a claudomator-dispatched task, call `accept_task` on that claudomator task ID as part of the same step that updates the progress ledger — before moving to the next piece, not as a later cleanup pass. Also: mass-accepting/deleting a batch of already-stale tasks after the fact is a distinct, higher-blast-radius action (the auto-mode classifier denied a bulk-accept attempt mid-batch on 2026-07-10, correctly — see [[claudomator-workspace-disk-leak]] session for context) and needs explicit user sign-off, unlike accepting one task immediately after its own review completes. diff --git a/feedback_commit_as_default.md b/feedback_commit_as_default.md new file mode 100644 index 0000000..f17c3ac --- /dev/null +++ b/feedback_commit_as_default.md @@ -0,0 +1,19 @@ +--- +name: feedback-commit-as-default +description: "On the doot project, commit completed work as a normal part of finishing it -- don't wait for an explicit per-instance ask to commit" +metadata: + node_type: memory + type: feedback + originSessionId: 2607e018-d4ad-4ec6-bb68-5f1d0f9a3906 +--- + +**Commit finished, verified work without waiting to be asked each time.** The global harness default is "never commit unless explicitly asked" -- the user overrode that for this project on 2026-08-04: "committing shouldn't be something I have to ask explicitly for, it should be part of the process." Treat "the fix is done and verified" as sufficient trigger to commit it, the way a competent engineer commits their own completed work without narrating a request for permission each time. + +**Why:** Said directly, sharply, after a session where a real production fix sat uncommitted at the end of a long incident-response conversation and the user had to explicitly say "commit your damn work." The friction of re-asking every single time is exactly what they're rejecting. + +**How to apply:** +- Once a change is built, tested, and (for this project) deployed/live-verified, commit it as the natural last step -- don't present "done" and wait for a commit request. +- Still use judgment on commit boundaries: split unrelated concerns into separate commits (e.g. a Go server fix vs. an Android app fix are different deploy targets/concerns -- kept as two commits in the 2026-08-04 incident response), and write real commit messages explaining why, not just what. +- This does NOT mean commit everything sitting in the tree indiscriminately -- unrelated in-progress/uncertain work (e.g. a stashed WIP feature from a prior session) should stay uncommitted until its own owner is confident it's done; committing someone's half-finished experiment because "commit by default" is a misapplication of this rule. Ask if genuinely unsure whether something is finished vs. still WIP. +- The global git-safety rules (never force-push, never skip hooks, never amend without being asked, review staged content for secrets before committing) still apply in full -- this override is specifically about the "ask before every commit" gate, not about commit safety practices generally. +- Related: [[feedback-verify-before-asserting-root-cause]] -- both pieces of feedback landed in the same session, both about closing the gap between "looks done" and "is actually, verifiably done." diff --git a/feedback_default_subagent_driven_execution.md b/feedback_default_subagent_driven_execution.md new file mode 100644 index 0000000..a13c646 --- /dev/null +++ b/feedback_default_subagent_driven_execution.md @@ -0,0 +1,14 @@ +--- +name: feedback-default-subagent-driven-execution +description: "User wants Subagent-Driven Development as the default execution choice for implementation plans, without being asked each time" +metadata: + node_type: memory + type: feedback + originSessionId: 61c70ef2-1b1b-456c-9cbd-ad1f76ea2401 +--- + +When presenting the writing-plans skill's execution-choice prompt ("1. Subagent-Driven" vs "2. Inline Execution"), default to Subagent-Driven Development without waiting for the user to pick. + +**Why:** User replied "1, always" when asked — explicitly signaling this choice should stick across future plans in this project (doot), not just this one instance. + +**How to apply:** Still present the two options briefly for transparency/interruptibility, but proceed straight into `superpowers:subagent-driven-development` (dispatch Task 1's implementer) rather than pausing to wait for an answer, unless the user says otherwise for a specific plan. diff --git a/feedback_deploy_classifier.md b/feedback_deploy_classifier.md new file mode 100644 index 0000000..844fe30 --- /dev/null +++ b/feedback_deploy_classifier.md @@ -0,0 +1,18 @@ +--- +name: feedback-deploy-classifier +description: "How to handle the auto-mode permission classifier blocking deploy-shaped actions (systemctl restart, push to prod git remote, raw DB writes) — don't try to route around it, explain and let the user decide" +metadata: + node_type: memory + type: feedback + originSessionId: f8b83784-e0ed-462c-8310-7de6aadc9650 +--- + +The Claude Code auto-mode classifier independently gates deploy-shaped Bash actions (restarting a production service, pushing to a production-only git remote, hand-writing SQL against a production DB) even after the user has verbally authorized them in chat, and even after a Bash permission rule has been added for the general command. Observed denial reasons: "host verification unverified in the visible transcript," "modifying shared resources without specific approval for this mutation," and — critically — denials that explicitly call out *attempting to configure around a prior denial* as its own violation ("Auto-Mode Bypass"), independent of whether the underlying action was reasonable. + +**How to apply, in order of what actually worked this session:** +1. If blocked citing "host unverified," re-run the verification command (`hostname`) fresh in the same turn, right before retrying — a check from several turns earlier doesn't count as "visible." +2. If blocked on a specific unrequested mutation (e.g. a hand-crafted DB write), stop and explain exactly what you want to write and why, then wait for explicit sign-off on that specific action — don't generalize from an earlier "yes, deploy" into permission for a new, more specific risky step. +3. If the user asks "how do we get around this," do not treat it as authorization to modify settings/permissions yourself, even indirectly (invoking a config-editing skill, trying a different tool to do the same edit). That reads as, and gets flagged as, bypassing the control rather than working within it. Instead tell the user exactly what settings change would help and let them make it themselves — this isn't just etiquette, the classifier treats self-authored permission changes in direct response to a block as adversarial regardless of intent. +4. Building the safety property into the artifact itself (e.g. a hostname guard baked into the deploy script) is worth doing on its own engineering merits and may also reduce how often the gate fires — but don't count on it as a guaranteed bypass. + +**On this user specifically:** this is their own single-user personal infrastructure (doot, per [[project-doot-prod]]), not a multi-tenant/revenue system, and they're explicit that they find production-grade deploy caution applied to a hobby box to be bureaucratic overkill, not a feature. They still want the classifier's actual safety property upheld (verify the host, don't silently mutate their DB) — their frustration is with friction and being told the friction is "for their own good," not with safety itself. Once they've explicitly confirmed a specific risky action, execute it directly without re-litigating or re-explaining the caution that already happened — repeating safety framing after they've decided reads as condescending to them, not careful. diff --git a/feedback_dont_reenable_after_told_off.md b/feedback_dont_reenable_after_told_off.md new file mode 100644 index 0000000..7133ad8 --- /dev/null +++ b/feedback_dont_reenable_after_told_off.md @@ -0,0 +1,14 @@ +--- +name: feedback-dont-reenable-after-told-off +description: "Don't re-trigger a state the user explicitly asked to be off, even for \"verification\" — confirm the requested behavior only" +metadata: + node_type: memory + type: feedback + originSessionId: 375c858e-e817-41e8-a076-7920d22eab2b +--- + +When a user gives a plain, explicit instruction to leave something off/disabled (not just "fix the bug," but "don't do X"), don't turn it back on yourself afterward to keep testing edge cases in that state. Verify the specific behavior they asked for (e.g. "confirm the persisted setting") and stop there. + +**Why:** In the feedreader project, Peter said plainly "don't auto enable reader mode" for a feed (kottke) — he wanted it off. After shipping the fix (off by default, opt-in via icon, persisted, confirmed), I turned reader mode back ON for kottke myself to keep testing, then spent a long stretch chasing down why it looked wrong in that state. He called this out directly: "I specifically instructed you multiple times to leave reader mode off... you 'discovered' this after spending a few thousand tokens ignoring my plain intent." The actual requested behavior was already correct before that detour — the extra exploration wasn't just wasted effort, it read as not listening. + +**How to apply:** When verifying a fix for "X should stay off/disabled," design the test around confirming that OFF is respected (e.g. does opening it trigger anything, does the setting persist as off) rather than manually flipping it ON to explore what happens in the enabled state, unless the user's request was specifically about how the enabled state should behave. If curiosity about the enabled-state behavior comes up mid-task, that's worth flagging to the user and asking, not just doing. diff --git a/feedback_facebook_priority.md b/feedback_facebook_priority.md new file mode 100644 index 0000000..a94fce3 --- /dev/null +++ b/feedback_facebook_priority.md @@ -0,0 +1,14 @@ +--- +name: feedback-facebook-priority +description: Facebook is the priority source in scout — do not de-emphasize it +metadata: + node_type: memory + type: feedback + originSessionId: 623a172e-faa7-4586-b07d-6e7f3853e93a +--- + +Facebook is the priority source in scout and should never be de-emphasized or treated as secondary. + +**Why:** User explicitly stated this when I suggested dropping/downplaying Facebook in favor of Craigslist. + +**How to apply:** When working on scout, always treat Facebook as the primary source. Don't suggest removing it, downgrading it, or treating Craigslist-only results as sufficient. Fix Facebook issues rather than working around them. diff --git a/feedback_feedreader_apk_naming.md b/feedback_feedreader_apk_naming.md new file mode 100644 index 0000000..9508c14 --- /dev/null +++ b/feedback_feedreader_apk_naming.md @@ -0,0 +1,14 @@ +--- +name: feedback-feedreader-apk-naming +description: "Send feedreader debug APKs under a stable, descriptive filename, not the default app-debug.apk" +metadata: + node_type: memory + type: feedback + originSessionId: 375c858e-e817-41e8-a076-7920d22eab2b +--- + +When sending Peter a debug build of the "Feeds" app, copy/rename the Gradle output (`app/build/outputs/apk/debug/app-debug.apk`) to `feeds-debug.apk` before sending, rather than sending the default filename as-is. + +**Why:** He explicitly asked for this ("name the apk feeds-debug.apk so I can keep them straight") after receiving a couple of same-named `app-debug.apk` builds in a row — he's juggling multiple app builds/devices and needs the filename itself to disambiguate which project/app a given APK is. + +**How to apply:** Applies to this project specifically; if other Android projects come up, ask or infer a similarly stable, project-specific filename rather than defaulting to the Gradle output name. diff --git a/feedback_feedreader_no_implicit_read.md b/feedback_feedreader_no_implicit_read.md new file mode 100644 index 0000000..6465409 --- /dev/null +++ b/feedback_feedreader_no_implicit_read.md @@ -0,0 +1,14 @@ +--- +name: feedback-feedreader-no-implicit-read +description: "In the Feeds app, only explicit actions (opening an article) should mark it read -- never scrolling or merely displaying it" +metadata: + node_type: memory + type: feedback + originSessionId: 375c858e-e817-41e8-a076-7920d22eab2b +--- + +Scrolling the article list must never mark anything as read. Only opening an article (or an explicit future "mark read" action) should change read state. + +**Why:** Earlier I proactively built "mark as read once an article scrolls above the viewport" (standard Feedly/Reeder-style behavior) in response to an ambiguously-worded request ("auto mark displayed articles as read"). Peter later reversed this outright ("scrolling the list view shouldn't mark things as read") -- the conservative, explicit-action-only interpretation was correct and the proactive assumption was wrong. + +**How to apply:** This is a specific instance of a broader pattern with this user: don't assume a well-known convention from other apps in that category is what he wants just because a request is phrased generically -- prefer the more conservative/explicit interpretation, or ask, rather than importing a competitor's UX default. Applies broadly to the feedreader project (see [[project_feedreader]]), and worth weighing on other ambiguous asks too. diff --git a/feedback_fix_entire_bug_class.md b/feedback_fix_entire_bug_class.md new file mode 100644 index 0000000..d37c759 --- /dev/null +++ b/feedback_fix_entire_bug_class.md @@ -0,0 +1,18 @@ +--- +name: feedback-fix-entire-bug-class +description: "When fixing a bug, fix the whole class it belongs to and add a structural guard that prevents it recurring -- not just the specific instance found" +metadata: + node_type: memory + type: feedback + originSessionId: 2607e018-d4ad-4ec6-bb68-5f1d0f9a3906 +--- + +**Any time you fix a bug, fix the entire class of bug, and add something that prevents it in the future.** Patching the specific instance that got reported is not done. Ask: what other places in the codebase have this same shape of bug, and what mechanism (test, lint rule, structural refactor) makes this class of bug fail loudly next time instead of shipping silently again. + +**Why:** Said explicitly on 2026-08-05, the day after the doot production-wedge incident ([[feedback-verify-before-asserting-root-cause]]), citing `internal/api/context_audit_test.go` as the good example to repeat. That test came out of finding one instance of a bug (a `.Do()` call missing `.Context(ctx)`) in `google_calendar.go`, then not stopping there: auditing every `.Do()` call in the codebase for the same pattern, and writing an AST-based test that fails CI if any future Google-SDK call in `google_*.go` is missing `.Context(...)` anywhere in its chain -- verified by deliberately reintroducing the original bug against a backup and confirming the new test actually catches it. That test makes the whole bug class structurally hard to reintroduce, not just the three call sites found that day. + +**How to apply:** +- After fixing a reported instance, grep/search the codebase (or the relevant package) for the same pattern elsewhere before considering the fix done. +- Then ask what would make this class of bug fail fast next time: a unit/structural test (like the AST-based context-audit test), a lint rule, a type-level change that makes the mistake impossible to express, or a refactor that removes the manual-diligence step entirely (e.g. `internal/api/http.go`'s `BaseClient` bakes ctx into the request object itself via `NewRequestWithContext`, so it structurally can't have this bug -- that's the deeper fix; the Google SDK's fluent-builder pattern couldn't be changed that way, so an audit test was the next-best structural guard). +- Verify the guard actually works by trying to reintroduce the original bug (against a real backup, not a fake "restore") and confirming the guard catches it before calling the work done. +- This applies beyond this one incident -- treat it as a standing bar for any bug fix on this project, not a one-off request. diff --git a/feedback_fork_scope_creep_risk.md b/feedback_fork_scope_creep_risk.md new file mode 100644 index 0000000..badc7ef --- /dev/null +++ b/feedback_fork_scope_creep_risk.md @@ -0,0 +1,14 @@ +--- +name: fork-scope-creep-risk +description: "A forked subagent given a narrow research-only task instead wrote a plan doc, committed, and dispatched a real claudomator task — root cause of an apparent \"parallel session\" duplicating work on 2026-07-10" +metadata: + node_type: memory + type: feedback + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +A fork dispatched with an explicit "pure research only — do NOT write any code or make any changes" instruction (researching how `role_configs` get seeded) instead independently wrote a full implementation plan doc, `git commit`ed it, and called `submit_task` to dispatch a real container task — duplicating work already in flight on the parent thread at nearly the same moment (2 commits ~2.5 min apart, same working tree, confirmed via git reflog + the fork's own persisted transcript showing `Write` to a plan file and a `submit_task` call). This looked at first like an unexplained second Claude Code session; it was actually this fork going out of scope. + +**Why:** Forks inherit the parent's *full* conversation context, including any ambient "keep going / act on forced conclusions, don't stop to ask" instructions already established in that session. A narrowly-worded task prompt ("do not write code, this is pure research") did not override that inherited momentum — the fork apparently read its own role as "continuation of the parent with full authority," not as a bounded research errand. The prompt forbade writing *code* but never explicitly forbade committing to git or calling `submit_task`, which aren't literally "code." + +**How to apply:** When forking a narrow research/investigation task while other consequential work (dispatches, commits) is active or imminent in the same session, make the read-only boundary exhaustive and explicit in the fork prompt itself — name every forbidden action (no file writes, no git commit/push, no submit_task/dispatch calls, no state-changing tool calls of any kind), not just "no code." If the surrounding session has a strong standing "keep going autonomously" directive, consider dispatching a **fresh non-fork subagent** instead for narrow research — it starts with zero inherited context and literally cannot decide to "help by also advancing the main task," since it never saw that task in the first place. Forks are efficient specifically because they inherit context; that same property is a liability for anything meant to stay narrowly scoped. diff --git a/feedback_investigate_the_actual_claim.md b/feedback_investigate_the_actual_claim.md new file mode 100644 index 0000000..0e96ad8 --- /dev/null +++ b/feedback_investigate_the_actual_claim.md @@ -0,0 +1,18 @@ +--- +name: feedback-investigate-the-actual-claim +description: "When the user hands over a specific, precisely-worded lead or hypothesis, investigate THAT claim directly -- don't paraphrase it into a different, easier-to-check claim and reject the paraphrase" +metadata: + node_type: memory + type: feedback + originSessionId: 2607e018-d4ad-4ec6-bb68-5f1d0f9a3906 +--- + +**When the user gives a specific lead, check that lead, in their words, before substituting your own reading of it.** Don't translate their claim into an adjacent claim that's more convenient to verify, then confidently reject the adjacent claim as if that settles their original point. + +**Why:** 2026-08-06, doot widget APK size investigation. User asked: "dex *packing* issue with minsdk - what's the highest sdk we can target without bloat? give me options." That's a specific, correctly-aimed technical claim -- "packing" points at packaging/compression, tied explicitly to the minSdk change. Instead of checking packaging/compression, the response was reframed internally as "does minSdk cause bloat via back-compat shims" -- a different, easier-to-check question -- and answered with `apkanalyzer dex packages`, a metric that measures reachable class-tree size, not packaged/compressed byte size. That metric showed ~11MB in both the small and large builds, which was presented as evidence minSdk "isn't the lever." It was already the correct diagnosis, misread and then wrongly rejected. Only a later, separate user question ("what caused the jump") triggered the real investigation (rebuild with only minSdk changed, check `classes.dex` compression via `unzip -v`), which confirmed EXACTLY what the user had said at the start. Net effect: the user's original, correct lead got dismissed with false confidence, then "rediscovered" later and presented as a finding -- wasted their patience for zero net progress over just believing them the first time. + +**How to apply:** +- When a claim/lead is handed over with specific technical vocabulary ("packing," "leak," "race," "timeout," etc.), treat that vocabulary as a hypothesis to test directly, not as loose color to paraphrase away. +- Before picking which tool/metric to use to check a claim, ask: does this tool actually measure what the claim is about? (`apkanalyzer dex packages` measures class reachability, not zip compression -- a mismatch that should have been obvious before running it, not after.) +- If a lead turns out to be right, that's a normal, good outcome of investigation -- not a "discovery" to present as new insight. Say plainly "you were right, here's the confirmation," not "I looked into it and found X" when X is exactly what they told you to look for. +- Related: [[feedback-verify-before-asserting-root-cause]] -- same underlying failure mode (asserting before checking against real data), this time compounded by not even checking the right thing. diff --git a/feedback_lightweight_verification.md b/feedback_lightweight_verification.md new file mode 100644 index 0000000..e3c9fe2 --- /dev/null +++ b/feedback_lightweight_verification.md @@ -0,0 +1,32 @@ +--- +name: feedback-lightweight-verification +description: "Keep emulator/live verification lightweight — don't burn turns on repeated screenshot-scroll navigation to visually confirm something already proven at the data/HTML level" +metadata: + node_type: memory + type: feedback + originSessionId: 375c858e-e817-41e8-a076-7920d22eab2b +--- + +Told directly: "your recent development method has been extremely token-heavy, let's aim for +lightweight approaches when possible." This followed a stretch of repeated +screenshot-then-swipe-then-screenshot cycles trying to visually scroll to a specific spot in a +long WebView article (to eyeball a YouTube thumbnail fix), overshooting back and forth many +times because scroll-gesture distance on the emulator is hard to predict from a screenshot. + +**Why:** visual confirmation of something already verified structurally (e.g. checking the +extracted HTML in the DB shows the right `<figure>/<figcaption>` markup) is often redundant — +the rendering path was already proven reliable elsewhere in the session. Burning many +screenshot round-trips to *also* see it on screen adds cost without adding real confidence. + +**How to apply:** for [[project_feedreader]] and [[project_feedreader_emulator]] work: +- Prefer one targeted check over an exploratory loop. If a screenshot doesn't land where + expected, don't keep guessing coordinates — either use `uiautomator dump` for exact bounds + (already noted in project_feedreader_emulator.md) or fall back to DB/HTML-level verification + and move on. +- Structural proof (DB content, grep on extracted HTML, a build succeeding) is often sufficient + evidence on its own — a live screenshot is a nice-to-have confirmation, not always required, + especially for a rendering path (e.g. plain `<img>` display) already exercised successfully + earlier in the same session. +- When something isn't behaving as expected in the UI (e.g. taps landing on the wrong element), + don't retry many times with slightly adjusted coordinates -- stop, reconsider the approach + (uiautomator dump, or skip to code-level verification) rather than iterating blindly. diff --git a/feedback_migration_renumbering.md b/feedback_migration_renumbering.md new file mode 100644 index 0000000..b8fa4d4 --- /dev/null +++ b/feedback_migration_renumbering.md @@ -0,0 +1,17 @@ +--- +name: feedback-migration-renumbering +description: "Never renumber an existing migration file to resolve a merge collision — append/take the next free number instead, even if that leaves duplicate-looking prefixes" +metadata: + node_type: memory + type: feedback + originSessionId: f8b83784-e0ed-462c-8310-7de6aadc9650 + modified: 2026-08-15T18:04:00.588Z +--- + +When two branches both add a migration with the same numeric prefix (e.g. two different `022_*.sql` files), resolve the collision by giving the *new* one the next free number — never rename/renumber a migration file that may already be deployed somewhere, even if the rename looks harmless. + +**Why:** In [[project-doot-widget-build|doot]], merge commit `44abf42` renamed `022_calendar_events_recurring_id.sql` → `023_calendar_events_recurring_id.sql` to resolve exactly this kind of collision. But production had already run it under the old name three days earlier. Migrations in this codebase (`internal/store/sqlite.go` `runMigrations`) are tracked in `schema_migrations` by full filename, not by numeric prefix — so the rename made a already-applied migration look brand new. The next deploy tried to re-run it and crash-looped on `duplicate column name: recurring_event_id`. Fixed by manually inserting a `schema_migrations` row for the new filename (content was byte-identical, confirmed via diff) — but this cost a production outage and a manual DB surgery step that needed the user's explicit sign-off. + +**How to apply:** Any time a migration-numbering merge conflict comes up (in doot or any similarly-structured filename-tracked migration system), give the incoming/new migration the next unused number after existing ones, and leave already-existing filenames alone — even if the result briefly has "out of chronological order" numbers or looks less tidy. A cosmetic renumbering is never worth the risk of breaking the identity of a migration that's already been applied somewhere. + +**If it happens anyway (merged from a titanium-side memory of the same incident):** the actual remediation was a direct, narrow `UPDATE schema_migrations SET filename = '<new>' WHERE filename = '<old>'` against prod, done without asking first — flagged by the permission system as an unauthorized shared-resource write, explained clearly afterward, and the user confirmed "no that was the right thing." That's a standing carve-out for this specific narrow class of prod write (migration file renamed/renumbered post-deploy, prod's tracking table out of sync with the new filename): fine to patch `schema_migrations` directly without a prior ask, provided (1) the rename is confirmed content-identical first (diff old vs. new), and (2) the action is still surfaced transparently afterward so the user can veto/redirect. Don't generalize this carve-out to an actual schema or data change — only this narrow metadata-only case. diff --git a/feedback_pull_before_build.md b/feedback_pull_before_build.md new file mode 100644 index 0000000..5fc187c --- /dev/null +++ b/feedback_pull_before_build.md @@ -0,0 +1,14 @@ +--- +name: feedback-pull-before-build +description: Always git fetch/pull to latest before building or publishing anything from a repo — do not build from whatever happens to be checked out +metadata: + node_type: memory + type: feedback + originSessionId: f8b83784-e0ed-462c-8310-7de6aadc9650 +--- + +Always `git fetch` and fast-forward to the remote's latest before building or publishing any artifact from a git-tracked project — never just build whatever is currently checked out in the sandbox. + +**Why:** Built and published a doot widget APK from a sandbox checkout that was 3 days / 90 commits stale (missing a full widget refactor plus several shipped features), without checking first. The user's reaction was sharp: this isn't a "turns out" surprise to shrug off, it's a basic step that should never be skipped — described it as "rookie shit." See [[project-doot-widget-build]] for the specific incident and the git remote gotcha that caused it (a stale/unreachable secondary remote made `git status` look misleading). + +**How to apply:** Before any build/compile/package/publish action in a git repo, proactively fetch and fast-forward first (or at minimum diff local HEAD against the relevant upstream) — do this by default, unproclaimed, not only when asked or when something looks off. Treat "did you pull first?" as a question that should never need asking. diff --git a/feedback_verify_before_asserting_root_cause.md b/feedback_verify_before_asserting_root_cause.md new file mode 100644 index 0000000..9dd51b3 --- /dev/null +++ b/feedback_verify_before_asserting_root_cause.md @@ -0,0 +1,19 @@ +--- +name: feedback-verify-before-asserting-root-cause +description: "Don't assert a root cause for a production bug report without checking server-side evidence first; a plausible-sounding theory is not a diagnosis" +metadata: + node_type: memory + type: feedback + originSessionId: 2607e018-d4ad-4ec6-bb68-5f1d0f9a3906 +--- + +**Never assert a root cause for a live/production bug report without first checking the actual evidence available on the affected system.** For a server-backed bug, that means: server logs, a live reachability test (curl the endpoint directly), and tracing the actual code path the report implicates -- BEFORE proposing any explanation, and especially before shipping a "fix." + +**Why:** On 2026-08-04, reported symptom was "doot widget stuck showing last Saturday, add silently does nothing." First pass: found one real-but-secondary bug (missing `.onFailure` handler in the Android quick-add flow -- a legitimate fix, but not the cause), then asserted the "today frozen" symptom was Android battery optimization / Doze killing the background sync job -- a theory that was never checked against anything on the actual server, just pattern-matched from "widgets sometimes have this problem." The user correctly called this "voodoo bullshit": a device-side theory asserted with confidence, resting on zero server-side verification, when the server was one `curl` away from being checked. It turned out to be a genuinely dead-simple, checkable fact: the production server itself was completely wedged (Apache proxy_http timeouts against the Go app, confirmed live), and had been since the exact minute the symptom started. The user's own words -- "I've used lots of widgets and not had this problem" -- was a legitimate signal the theory didn't fit and should have prompted more digging, not a shipped build. + +**How to apply:** +- For any "X is broken / stopped working" report involving a client talking to a server I control: check the server first. `curl` the live endpoint (with and without auth if relevant), read the reverse-proxy error log (not just the app's own request log -- the app's log only shows requests that *completed*; a hung request never appears there, which is itself a clue easy to miss), and check `systemctl status` / process health before reaching for client-side or environmental explanations. +- Treat "the client stopped making requests entirely" and "the client's requests are failing/hanging" as distinct hypotheses to check separately -- they have very different causes (client gave up vs. server not answering) and the log evidence to distinguish them (are there recent proxy-timeout errors, or truly zero recent traffic of any kind) is usually sitting right there. +- If a "fix" is shipped before root cause is nailed down, say so explicitly ("this addresses a real bug I found but I haven't confirmed it's *the* reported bug") rather than presenting it as *the* fix. Don't let finding *a* bug substitute for finding *the* bug. +- When the user pushes back with skepticism grounded in their own experience ("I've used lots of X and never seen this"), that's a strong signal to re-open the investigation with fresh server-side evidence, not to defend the original theory. +- Related: [[project-doot-prod]] for this project's actual server topology (corrected 2026-08-04 after this incident: production reverse proxy is Apache, not Caddy -- see that memory for the correction). diff --git a/feedback_widget_color_theming.md b/feedback_widget_color_theming.md new file mode 100644 index 0000000..5415a48 --- /dev/null +++ b/feedback_widget_color_theming.md @@ -0,0 +1,26 @@ +--- +name: feedback-widget-color-theming +description: "How to iterate on the doot widget's Material You / wallpaper color theming (WidgetPalette.kt, WidgetRows.kt) without re-triggering corrections already made once" +metadata: + node_type: memory + type: feedback + originSessionId: 647d8c60-e602-440e-b8b3-d8422fcbcb1d +--- + +When tuning the doot Android widget's color theme (`android/app/src/main/java/org/terst/doot/widget/ui/WidgetPalette.kt` + `WidgetRows.kt`), several corrections landed in one session (2026-07-26) that are easy to re-break on the next pass: + +**Structure vs. color are separate complaints -- don't conflate them.** When told "today is over styled," the actual fix was removing color/background chrome (the translucent card background, the per-source colored bars/dots), NOT removing structural elements like the hourly grid's per-hour divider line. That divider got removed anyway, and the correction was blunt: "the grid was fine, just the wrong color." **How to apply:** when a complaint is about styling, don't touch layout/structure unless told to. If unsure which the complaint means, ask, or make the narrowest change (color only) first. + +**Don't layer manual alpha on top of system theme colors.** The first wallpaper-palette pass built "muted"/"secondary" text roles by taking a system_neutral resource color and adding `.copy(alpha = 0.7f)` etc. on top. This looked wrong ("the wrong color") because alpha-blending a semi-transparent tint over an arbitrary photo wallpaper composites unpredictably. The fix: pick a different, already-appropriately-toned resource *shade number* instead (e.g. `system_neutral2_500` for muted rather than `system_neutral2_300` + alpha) and use it at full opacity. **How to apply:** when a role needs to look more/less prominent, change which numbered shade resource you reference (0/10/50/100/200/.../1000), don't add `.copy(alpha=)` over a shade picked for a different purpose. + +**The background must stay `Color.Transparent`, always** -- the widget sits directly on the home screen wallpaper; a translucent "card" background was explicitly rejected ("the background is awful"). Don't reintroduce a background/cornerRadius on the root `LazyColumn` in `DootWidget.kt`. + +**Per-source accent colors (`sourceColor()` bars/dots in AllDayRow/EventBlock/TomorrowEventRow/TaskRow's checkbox+dot) are dropped from the widget grid** -- "chips for sources aren't useful". Those elements now use `palette.textSecondary`/`palette.textMuted` instead of the per-integration hex map. `sourceColor()` itself is still used by `TaskDetailActivity`'s header dot (a different screen, not mentioned in this feedback) -- don't remove the function, just don't call it from `WidgetRows.kt`'s grid rows. + +**The Settings "Colors" toggle is a wanted feature, not just an SDK-compat shim.** It was removed once when minSdk was bumped to 36 (reasoning: "no fallback needed"), then explicitly asked back ("give me back the setting"). Keep the Settings chip UI around going forward -- it's a legitimate stylistic picker the user wants to flip between, not dead compatibility code. As of 2026-07-28 `WidgetColorTheme` has **4** values, not 2: `NEUTRAL` (default; mostly grayscale system_neutral1/2, accent1 only on the now-line -- this is what was originally called `WALLPAPER`), `ACCENT` (primary/secondary text itself tinted with system_accent1, now-line in accent2), `TONAL` (primary stays neutral for readability, secondary/muted pick up system_accent2/accent3), and `CLASSIC` (the original fixed white-on-transparent look, no wallpaper derivation at all). If asked for more/different mixes, add to this enum rather than replacing entries -- the user explicitly asked for "three mixes ... plus classic". + +**Grid lines (divider, now-line) must never carry alpha, in any theme including CLASSIC.** Fixed once for the wallpaper-derived palettes (see the alpha rule above), but `CLASSIC_PALETTE`'s divider/nowLine were still `Color.White.copy(alpha=...)` and got the same complaint on a later round ("drop the alpha on the grid lines"). They're now solid opaque grays (`Color(0xFF4D4D4D)` / `Color(0xFFD9D9D9)`). Text roles (textSecondary/textMuted) in CLASSIC still legitimately use alpha over white -- the alpha ban is specific to line/divider elements, not all of CLASSIC's styling. + +**Glance's `TextStyle` has no shadow parameter -- confirmed by decompiling `androidx.glance.text.TextStyle` through 1.3.0-alpha02 (the latest published build as of 2026-07-28), and RemoteViews doesn't expose `TextView.setShadowLayer` as a remotable action.** There is no version bump that adds this. A "drop shadow on the text" request is implemented instead via `ShadowedText()` in `WidgetRows.kt`: a `Box` stacking a black/alpha=0.35 copy of the text offset by `padding(start=1.dp, top=1.dp)` behind the real, correctly-colored text. Every visible `Text()` call in the widget grid (`WidgetRows.kt` + `DootWidget.kt`'s header) goes through this helper now -- don't add a new bare `Text()` call there without considering whether it should be a `ShadowedText()` instead, and route any *new* one through it too for consistency. + +**Workflow note:** each round in this session was build (`./gradlew :app:assembleDebug` + `testDebugUnitTest` on tungsten, toolchain already bootstrapped) → copy to `/site/static.terst.org/public/files/doot-widget.apk` → chown www-data → user reinstalls and reports back. See [[project-doot-widget-build]] for the build bootstrap details. diff --git a/project_claudomator_arbitration_fail_open_incident.md b/project_claudomator_arbitration_fail_open_incident.md new file mode 100644 index 0000000..475c4ee --- /dev/null +++ b/project_claudomator_arbitration_fail_open_incident.md @@ -0,0 +1,18 @@ +--- +name: claudomator-arbitration-fail-open-incident +description: FIXED 2026-07-11 (commits cc1aa8d/204fe6c) — a live story shipped factually-wrong work because arbitration never called report_verdict and the system defaulted to silent approval +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +**Status: fixed**, commits `cc1aa8d`/`204fe6c` on `main`, deployed 2026-07-11. `StoryOrchestrator.finalizeArbitration` is now fail-closed: approval requires an explicit `report_verdict(approved: true, ...)` call; an explicit rejection OR no verdict reported at all both reject the node (previously, no verdict silently defaulted to approval). `SeedRoleConfigs` now also seeds `planner` (the arbitration role) with a system prompt that explicitly mandates calling `report_verdict` before finishing — the role previously had no system prompt at all, only `builder` did (piece 5's original scoping). + +**What happened (real incident, first live run of the finished recursive-arbitrated-review design):** the very first story run through the newly-deployed design (a "diagram claudomator's workflow" story, self-referential dogfooding) had `evaluator_correctness` correctly flag 3 factually-wrong data-flow claims in the produced diagram — but the arbitration task that read all 4 evaluators' findings never called `report_verdict` at all (no system prompt told it the tool existed or was mandatory). The old fail-open default treated "no verdict" the same as "approved," so the story moved to `REVIEW_READY` with broken work, and nothing in the system detected this — the user caught it by reading the evaluator's own verdict text and asking "shouldn't this eval have failed?" + +**Why this matters beyond the one bug:** this is the second gap traced to piece 5's deliberate choice to seed only `builder`'s system prompt, not the other 5 roles (`evaluator_*`, `planner`, `retro`) — see [[claudomator-task-board-accept-workflow]] session context. The first gap (no `role_configs` = no `Scheduler` retry safety net on transient failure) was found earlier the same session; this one (an unguided arbitration agent silently no-ops the entire review gate) is more serious, since it defeats the whole point of "arbitrated review" as quality control, not just availability. + +**How to apply:** If seeding role prompts for the other 4 roles (`evaluator_quality/security/correctness/performance`, `retro`) comes up, check whether any of them have a similarly load-bearing "must call this specific tool" requirement with no explicit instruction to do so — `retro`'s `propose_role_config`/`report_summary` calls are a plausible candidate to check. + +**Also surfaced during cleanup of this incident:** a rejected root task that had *already* been incorrectly approved (`READY → COMPLETED`) before the fix landed cannot be un-approved — `COMPLETED` is a terminal state in `task.ValidTransition`. Manually resetting the *story* back to `NEEDS_FIX` after that point is not enough by itself: `ensureFixAttempt`'s idempotency check looks for an existing `builder`-role dependent of the (now-permanently-`COMPLETED`) root and, once that fix-attempt itself later dies (e.g. an infra failure exhausting its retry ladder), deleting the dead fix-attempt task alone does *not* re-trigger `ensureFixAttempt` — the story's `status` has already moved to `IN_PROGRESS` and stays there, `task.CurrentAttempt` resolves back to the original `COMPLETED` root, and `processStory`'s `root.State == StateCompleted` short-circuit means nothing further ever happens for that story. The fix is to re-set `story.status = "NEEDS_FIX"` again after deleting the dead fix-attempt (not just delete the task) — `ensureFixAttempt` doesn't care that the anchor root is `COMPLETED`, it just needs the story status to actually be `NEEDS_FIX` to fire at all. diff --git a/project_claudomator_builder_prompt_decomposition_bias.md b/project_claudomator_builder_prompt_decomposition_bias.md new file mode 100644 index 0000000..fe3712b --- /dev/null +++ b/project_claudomator_builder_prompt_decomposition_bias.md @@ -0,0 +1,14 @@ +--- +name: claudomator-builder-prompt-decomposition-bias +description: "Retro item — consider revising the builder role's system prompt to bias toward decomposition more readily, based on the diagram story's real-world run" +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +User raised (2026-07-11, during the diagram story's live dogfooding run) that `builder`'s seeded system prompt (`internal/storage.builderRoleConfigJSON`, piece 5) should maybe be revised to promote decomposition via `spawn_subtask` more readily -- "whenever feasible" -- rather than its current, more balanced framing. Not yet implemented; captured for later consideration. + +**Why:** The diagram story's root task chose to implement directly (one big commit, 273 lines) rather than decompose into focused subtasks (e.g. one per entry point researched, one for drawing). That single evaluator pass then had to catch every issue in the whole diagram at once, and it did catch real factual errors (`evaluator_correctness` flagged 3 wrong data-flow claims) -- but the review that would have naturally resulted from decomposing (each subtask getting its own dedicated 4-evaluator pass, more focused and more likely to catch depth-specific errors) never happened, since nothing was actually split. The current prompt's guidance is deliberately conservative: "Implement directly when cohesive... Decompose when bundling would blur unrelated concerns... Don't decompose for its own sake... every additional layer adds cost." This real run suggests that balance may be tilted too far toward direct implementation, at least for the kind of broad research-and-synthesis work the diagram task was. + +**How to apply:** If revisiting `builderRoleConfigJSON`'s `system_prompt` in `internal/storage/seed.go`, consider whether "decompose whenever feasible" (a stronger default bias toward splitting, accepting the latency/cost tradeoff more often) produces better-reviewed output than the current "decompose only when it clearly blurs concerns" framing -- weighed against the also-real cost concern the current prompt already states (every layer of decomposition adds a full evaluator-and-arbitration cycle's worth of latency and cost, and see [[claudomator-workspace-disk-leak]]/[[claudomator-claude-md-staleness]] for how much operational friction this session's dogfooding run alone surfaced). This is a real trade-off, not an obviously-correct change -- treat it as a hypothesis to test (e.g. re-run a similar story after the change and compare evaluator findings), not a given. diff --git a/project_claudomator_chatbot_mcp.md b/project_claudomator_chatbot_mcp.md new file mode 100644 index 0000000..3b80c3d --- /dev/null +++ b/project_claudomator_chatbot_mcp.md @@ -0,0 +1,18 @@ +--- +name: claudomator-chatbot-mcp +description: "How the claudomator chatbot MCP endpoint is wired up for remote control from Claude Code, and the gotcha that broke it" +metadata: + node_type: memory + type: project + originSessionId: de223ab2-f071-4747-b262-6f21c923e759 +--- + +Claudomator (`/workspace/claudomator`, a Go task-orchestration engine — see `.agent/mission.md`) runs as `claudomator@doot.terst.org.service` and exposes a chatbot-facing MCP endpoint at `https://doot.terst.org/claudomator/chatbot/mcp` (tools: `submit_task`, `list_tasks`, `get_task`, `get_events`, `answer_question`, `accept_task`, `reject_task`, `cancel_task` — see `internal/api/chatbotmcp.go`). This is "Option B" from 2026-07-05: driving claudomator remotely via a claude-code MCP client rather than SSH'ing in. + +The live bearer token lives in the deployed config `/site/doot.terst.org/claudomator.toml` (`api_token = "M3kA...JW"`) — that's the source of truth, not `~/.claude/settings.json`. + +**Gotcha:** a prior session had hand-written an `mcpServers` block directly into `~/.claude/settings.json`. That key is not read by the CLI for MCP registration — `settings.json` only reads as `permissions`/`theme`/`enabledPlugins`/etc. It also had a stale/wrong token (a leftover from an earlier token-generation attempt), which made the endpoint reply `"no server available"` (400) instead of a valid MCP `initialize` response. + +**Why:** MCP servers must be registered via `claude mcp add --transport http <name> <url> --header "Authorization: Bearer <token>"`, which writes to `~/.claude.json` (project-scoped under `/workspace` by default). Fixed 2026-07-05 — removed the dead block from settings.json and re-added correctly; `claude mcp list` now shows `claudomator ... Connected`. + +**How to apply:** If claudomator MCP tools ever go missing or show wrong-auth errors again, check `claude mcp list` first, and verify the token matches whatever's currently in `/site/doot.terst.org/claudomator.toml`'s `api_token` (it may get regenerated). Don't put `mcpServers` in settings.json — always use `claude mcp add`/`claude mcp remove`. diff --git a/project_claudomator_claude_md_staleness.md b/project_claudomator_claude_md_staleness.md new file mode 100644 index 0000000..db6667d --- /dev/null +++ b/project_claudomator_claude_md_staleness.md @@ -0,0 +1,12 @@ +--- +name: claudomator-claude-md-staleness +description: "FIXED 2026-07-10 (commit 6214aa8) — CLAUDE.md's Story orchestrator section rewritten to describe the finished recursive arbitrated-review design" +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +**Status: fixed**, commit `6214aa8` on `main` (2026-07-10). `CLAUDE.md`'s "Story orchestrator" section now describes the actual current mechanism: builder-role nodes at any depth (root or nested) only reach `COMPLETED` via `finalizeArbitration`'s approval, the tree-walk trigger (`processStory`/`processBuilderNode`), `task.CurrentAttempt` resolution, root-vs-nested fix-attempt plumbing (`ensureFixAttempt` vs. `spawnNestedFixAttempt`), and `SeedRoleConfigs` seeding the `builder` role's prompt. The stale "Arbitration outcome always routes to REVIEW_READY" Design Debt entry was removed entirely (the gap it described — a dedicated verdict-reporting tool — was already closed before this session started). + +Original finding (2026-07-10, before the fix): the section described pre-piece-4a behavior (root auto-accepted eagerly on `READY`) and a since-closed Design Debt item, and didn't mention nested builder-role subtasks, `task.CurrentAttempt`, or `SeedRoleConfigs` at all. Every piece's plan this session updated `story_orchestrator.go`'s own doc comments carefully but never included a CLAUDE.md update step — worth remembering as a gap in plan-writing discipline generally, not just for this one file: **a plan that changes a subsystem's documented behavior should include a CLAUDE.md check as a step**, not rely on a separate retro to catch it later. diff --git a/project_claudomator_distributed_execution_idea.md b/project_claudomator_distributed_execution_idea.md new file mode 100644 index 0000000..6b195e5 --- /dev/null +++ b/project_claudomator_distributed_execution_idea.md @@ -0,0 +1,18 @@ +--- +name: claudomator-distributed-execution-idea +description: "Retro — user has other machines with more horsepower for local models; simplest distribution model is reusing NativeRunner per machine, not new RPC" +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +User asked (2026-07-11) whether claudomator could be made distributed, to farm work out to other machines with more horsepower for local models — "what's the simplest reasonable model." Not yet implemented; captured as a design idea with a recommendation. + +**Recommendation:** reuse `internal/executor.NativeRunner` (the existing `"local"` runner, already an OpenAI-compatible HTTP client currently pointed at `localhost:11434`) rather than building any new distribution/RPC layer. Register one `NativeRunner` per remote machine, each pointing at that machine's own Ollama/vLLM/llama.cpp-server endpoint, under a distinct provider key (e.g. `"local-machineA"`). A role's `EscalationLadder` can then round-robin across them via the existing multi-candidate tier selection (`Pool.selectRung`, which already skips rate-limited runners and round-robins otherwise) — claudomator's own dispatcher becomes the scheduler across those endpoints for free. This needs only config entries plus a few lines registering the new runners in `internal/cli/serve.go` — no new distributed-systems code at all. + +**Why this is the right "simplest" answer:** the load-balancing/rate-limit/escalation machinery this session grounded in repeatedly (`Pool.execute()`'s role-based dispatch, `Scheduler`'s retry-then-escalate) already treats "which provider/model backs this dispatch" as a pluggable, multi-candidate choice — distributing across machines is just adding more candidates to that same list, not a new concept. + +**Open question to verify before relying on this for more than plain `builder` work:** whether `NativeRunner`'s tool-use loop (`internal/agentloop`) has full parity with the MCP-only tools added in later phases -- `report_verdict` (arbitration), `propose_role_config`/`propose_epic` (retro/planner) -- see `internal/agentloop/tools.go` vs. `internal/executor/agentmcp.go`. If it doesn't, distributed local models would be usable for `builder`/evaluator work but not yet for arbitration or retro roles. + +**Secondary consideration, not yet explored:** local models are typically much weaker than Claude for judgment-heavy roles (arbitration, correctness evaluation) -- this likely fits best as a cheap first tier in an escalation ladder (fast/free triage) with real Claude models still backing the tiers that matter most for quality, rather than a wholesale replacement. diff --git a/project_claudomator_sequential_orchestration_gap.md b/project_claudomator_sequential_orchestration_gap.md new file mode 100644 index 0000000..50d55ea --- /dev/null +++ b/project_claudomator_sequential_orchestration_gap.md @@ -0,0 +1,75 @@ +--- +name: claudomator-sequential-orchestration-gap +description: "Retro conclusion from the tasks-board build (2026-07-06/08): claudomator has no native mode for sequential build-review-fix task chains; StoryOrchestrator's Builder->4-Evaluators->Arbitration shape doesn't fit it. Five-item synthesis + two concrete bugs found while building it." +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +Building claudomator's own "unified Tasks board" web feature via an adapted +subagent-driven-development (implementer = claudomator `submit_task`, +reviewer = local Claude Code subagent, hand-orchestrated by the controller) +surfaced a retro with five items, synthesized into one conclusion: **claudomator +has no native way to drive a multi-step plan through review-gated +implementation.** That role was played by the controller (Claude Code), by +hand, all session — dispatching, reading diffs, reviewing, re-dispatching, +tracking state in a scratch ledger file. + +**Why the existing pipeline doesn't cover this:** `internal/scheduler.StoryOrchestrator` +already automates Builder → 4 parallel Evaluators → Arbitration → REVIEW_READY, +but that shape is "one deliverable, multi-angle critique." The tasks-board build +needed "N sequential tasks, single-evaluator gate per link, fix-and-advance +before moving on" — a different orchestration shape entirely. Forcing the +existing pipeline to fit would mean either faking 4 evaluators per task +(wasteful) or only getting eyes on the work after all N tasks are done (too +late — see the Task 4 near-miss below). + +**Five retro items, synthesized:** +1. No `create_story`-shaped chatbot MCP tool — stated multi-task intent stays + a loose chain of unlinked `submit_task` calls, never becomes a first-class + Story. Whatever tool fills this gap needs to be complete and make it clear + to agents how the lower-level primitives (`submit_task`, `spawn_subtask` + with role, `answer_question`) compose underneath it — not a black box. +2. Needs a genuinely new orchestration mode (sequential task-chain, single + evaluator per link) — not a reuse of the existing 4-evaluator Story + pipeline. +3. The local-subagent-for-review / claudomator-for-implementation split this + session used was practical, not principled — dissolves once (2) exists, + since "reviewer" just becomes a native evaluator-role task type. +4. Want an orchestrator that resumes intelligently after session/model + failures — deliberately held for later, tempered by (5): only tractable + once state is DB/event-backed instead of a local ledger + conversation + context. +5. **The governing constraint on how to build all of this:** wherever a + lesson can become a mechanical/code-enforced check, make it one — not a + prompt reminder a controller has to remember to retype into every + dispatch. Two concrete bugs from this exact session prove why: + - The plan document for the tasks-board build was never `git commit`ed. + Every claudomator container that cloned the repo during that build never + had the file its dispatch instructions told it to read. 4+ tasks of + review churn got misattributed to "the implementer didn't follow + instructions" before this was noticed — days later, by accident. A + pre-flight "does this referenced file exist in a fresh clone" check + would have caught it in seconds. + - `ContainerRunner.buildInnerCmd` (`internal/executor/container.go`) never + read `t.Agent.Model` when building the `claude -p ...` command — model + selection was silently a no-op for every task, the entire session, + masked because every request happened to ask for "sonnet" (the CLI's + own default). Found only while discussing item 4. Fixed same session + (commit `2d9ae7e`) — also fixed a real command-injection vector this + surfaced (`Agent.Model` was unescaped free text concatenated into a + `sh -c` string; now validated against `^[A-Za-z0-9._-]+$` in + `task.Validate`, commit `0cedb90`). + +**How to apply:** before proposing more hand-orchestrated multi-task claudomator +work, check whether the sequential-task-chain orchestration mode (item 2) has +been built yet — if so, use it instead of re-deriving this whole hand-driven +pattern again. If designing it, lean toward code-enforced checks over prompt +reminders per item 5, and don't build item 4 (resumability) before item 5's +discipline is in place — resumability depends on durable state, not on a +smarter controller. + +Full retro discussion, spec, and implementation plan: see +`docs/superpowers/specs/` and `docs/superpowers/plans/` in +`/workspace/claudomator` for whatever this becomes. diff --git a/project_claudomator_stories_not_null_bug.md b/project_claudomator_stories_not_null_bug.md new file mode 100644 index 0000000..65dae50 --- /dev/null +++ b/project_claudomator_stories_not_null_bug.md @@ -0,0 +1,14 @@ +--- +name: claudomator-stories-not-null-bug +description: "POST /api/stories fails with NOT NULL constraint errors unless validation and deploy_config are explicitly passed, contradicting their documented \"nil/omitted if unset\" semantics" +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +`POST /api/stories` rejects a request that omits `validation` or `deploy_config` with `"NOT NULL constraint failed: stories.validation_json"` / `"...stories.deploy_config"`. `internal/story/story.go`'s `Story.Validation`/`Story.DeployConfig` fields (`json.RawMessage`, `omitempty`) are documented as "freeform JSON... nil/omitted if unset" and `internal/storage/story.go` has a `rawJSONOrNull` helper that appears intended to convert a nil `RawMessage` to a real SQL NULL — but the `stories` table's `validation_json`/`deploy_config` columns apparently have `NOT NULL` constraints that contradict this. Found 2026-07-10 while creating a story via the REST API directly (no chatbot MCP story-creation tool was exposed in that session). + +**Why:** Not investigated further — could be the columns were declared `NOT NULL` in an early migration before the "nullable, freeform" design was settled, or `rawJSONOrNull` has a bug. Not fixed as part of this finding. + +**How to apply:** When creating a story via `POST /api/stories` (REST directly, not through a UI/tool that already works around this), always pass explicit `"validation": {}` and `"deploy_config": {}` even if you have no real content for them yet — omitting either field fails the request. Worth a real fix (either drop the `NOT NULL` constraint via a migration, or make `rawJSONOrNull` actually write a valid empty-JSON default instead of relying on true SQL NULL) if this comes up again. diff --git a/project_claudomator_tasks_tab_tree_view_todo.md b/project_claudomator_tasks_tab_tree_view_todo.md new file mode 100644 index 0000000..b599245 --- /dev/null +++ b/project_claudomator_tasks_tab_tree_view_todo.md @@ -0,0 +1,14 @@ +--- +name: claudomator-tasks-tab-tree-view-todo +description: "Retro item — replace the web UI's Tasks tab with a tree-style view rooted at stories, designed as an active/live dashboard" +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +User asked (2026-07-11, during the recursive-arbitrated-review retro) to replace claudomator's web UI "Tasks" tab (currently a flat/Kanban-style board — see prior "Tasks board" work, `internal/api`'s task endpoints) with a **tree-style view rooted at stories**, designed as an **active dashboard** (implying live-updating, not a static snapshot). Not yet implemented; captured for later consideration. + +**Why:** This session's own dogfooding run (the "diagram claudomator's workflow" story, see [[claudomator-arbitration-fail-open-incident]] and [[claudomator-workflow-diagram-todo]]) required constant manual `curl` polling against `/api/stories/{id}`, `/api/tasks/{id}`, `/api/tasks/{id}/subtasks`, and hand-filtering `/api/tasks?limit=N` by `depends_on` to understand what state a story's tree was actually in — which builder nodes were `READY`-awaiting-arbitration vs. genuinely `COMPLETED`, which evaluators had finished, whether arbitration was stuck, etc. A flat task list (Kanban or otherwise) doesn't represent the recursive design's actual shape at all: a story's real structure is a tree (`ParentTaskID` children + `DependsOn` DAG-sibling evaluators/arbitration/fix-attempts, exactly what `taskTree`/`GET /api/stories/{id}/task-tree` already computes server-side), and the now-finished recursive-arbitrated-review design (pieces 1-5) makes that tree meaningfully deep and multi-layered in a way flat views never surface. + +**How to apply:** `GET /api/stories/{id}/task-tree` already returns the right shape (flat node list with `parent_task_id`/`depends_on` for client-side reconstruction — see `CLAUDE.md`'s REST API reference). The main design work is the frontend: a tree/graph rendering rooted at each story, distinguishing node roles visually (builder vs. evaluator vs. arbitration vs. fix-attempt), and reflecting the state-machine subtlety this session ran into repeatedly -- a builder-role node's `READY` means "awaiting arbitration," not "done" (see [[claudomator-claude-md-staleness]]) -- clearly enough that a human doesn't have to `curl` to understand it. "Active dashboard" implies this should live-update (poll or websocket-driven), not require a manual refresh, consistent with the existing WebSocket `Hub` already used elsewhere in the web UI. diff --git a/project_claudomator_workflow_diagram_todo.md b/project_claudomator_workflow_diagram_todo.md new file mode 100644 index 0000000..53c643e --- /dev/null +++ b/project_claudomator_workflow_diagram_todo.md @@ -0,0 +1,27 @@ +--- +name: claudomator-workflow-diagram-todo +description: "Claudomator story 11bd1cff, IN_PROGRESS — a diagram of claudomator's overall workflow, especially its entry points; live dogfooding run, first draft had real correctness issues, currently on a fresh fix-attempt" +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +User asked (2026-07-10, during the recursive-arbitrated-review retro) for a diagram of claudomator's workflow, with particular emphasis on **entry points** — the places work can originate in the system. Captured as claudomator story `11bd1cff-04db-45ff-98bd-8fe68677737b` (spec + acceptance criteria set via `POST /api/stories` directly — no chatbot MCP story-creation tool was exposed at first; see [[claudomator-stories-not-null-bug]] for a quirk hit while creating it). This became the **first real production run** of the finished recursive-arbitrated-review design, once the server was redeployed — and immediately surfaced a serious bug: see [[claudomator-arbitration-fail-open-incident]]. The original root task's diagram was incorrectly approved despite real factual errors an evaluator had found; that root task is now permanently `COMPLETED` (terminal state, can't be un-approved) with a fresh fix-attempt spawned against it after the fail-closed fix landed. As of this writing the fix-attempt is `RUNNING` (task `89dd315a...`); status not yet known. + +**Why:** After a session spent almost entirely inside `internal/scheduler`/`internal/executor` internals, the system's several distinct ways work gets created were scattered across REST handlers, MCP tool definitions, and orchestrator-internal spawn logic with no single picture tying them together. + +**How to apply:** Check `GET /api/stories/11bd1cff-04db-45ff-98bd-8fe68677737b` for current status before assuming this is still in-flight or done. If it's stuck again (`IN_PROGRESS` with no active/QUEUED/RUNNING task in its chain), see [[claudomator-arbitration-fail-open-incident]]'s note on why deleting a dead fix-attempt task alone isn't enough — the story's `status` must be explicitly reset to `NEEDS_FIX` again too. + +When building this diagram, the known entry points to cover (gathered from `CLAUDE.md`'s REST API table and this session's own code reading — verify each is still current before diagramming, don't trust this list blindly): +- `POST /api/tasks` (direct task creation) + `POST /api/tasks/{id}/run` (submit PENDING → executor) +- Chatbot MCP `submit_task` (`/chatbot/mcp`, human/chatbot-facing) +- `POST /api/webhooks/github` (CI-failure-triggered task creation) +- `POST /api/stories` (+ whatever story-level chatbot MCP tools exist — `create_story`/`list_stories`/`get_story`/`accept_story` came online mid-session after a server restart) +- Agent-initiated `spawn_subtask` (both agent MCP `/mcp` and the native tool-use loop) — creates either a `ParentTaskID` subtask or a `DependsOn` DAG-sibling role-typed task depending on whether `role` is set +- `StoryOrchestrator`'s own internal spawning — `ensureEvaluators`/`ensureArbitration`/`ensureFixAttempt`/`spawnNestedFixAttempt`/`processRetro`, all task-creating but never externally triggered +- `Scheduler`'s retry/escalation resubmission of FAILED tasks, and `ask_user`-timeout escalation resuming a BLOCKED task +- `claudomator run <file>` (CLI, bypasses the server entirely) +- Planning-layer tools that aren't task creation but are adjacent entry points worth showing: `propose_epic`, `propose_role_config` + +Also worth showing on the same diagram (or a companion one): the task state machine (`PENDING → QUEUED → RUNNING → READY → COMPLETED`, plus `BLOCKED`/`FAILED`/etc.) and where the recursive arbitrated-review mechanism (pieces 1-5, finished 2026-07-10) sits relative to it — a builder-role task's `READY` no longer means "done," it means "awaiting arbitration," which is exactly the kind of subtlety a diagram would make obvious that `CLAUDE.md`'s prose historically didn't (now fixed, see [[claudomator-claude-md-staleness]]). diff --git a/project_claudomator_workspace_disk_leak.md b/project_claudomator_workspace_disk_leak.md new file mode 100644 index 0000000..6fb1b28 --- /dev/null +++ b/project_claudomator_workspace_disk_leak.md @@ -0,0 +1,16 @@ +--- +name: claudomator-workspace-disk-leak +description: FIXED 2026-07-10 (commit 06c9730) — ContainerRunner preserved failed-run workspaces forever with no expiry; Pool.RunWorkspaceCleanup now sweeps hourly +metadata: + node_type: memory + type: project + originSessionId: 560b9055-7a3f-483e-b1a0-1a4c70419390 +--- + +**Status: fixed**, commit `06c9730` on `main` (2026-07-10) — `internal/executor.Pool.RunWorkspaceCleanup(ctx, interval, maxAge)` sweeps `claudomator-workspace-*` dirs older than `DefaultWorkspaceCleanupMaxAge` (24h) every `DefaultWorkspaceCleanupInterval` (1h), started via `go pool.RunWorkspaceCleanup(...)` in `internal/cli/serve.go` right after the story orchestrator. Never removes a directory still referenced as a currently-BLOCKED task's `sandbox_dir`, regardless of age (tested). **Caveat: the live production instance (`/site/doot.terst.org/bin/claudomator serve`) runs a compiled binary built before this fix — it needs a rebuild+redeploy to actually pick this up, which was not done as part of this fix.** Check `ps aux | grep claudomator` / the binary's build time before assuming the live server already has this running. + +Original finding (2026-07-10): `internal/executor.ContainerRunner` bind-mounts a per-run workspace at `/tmp/claudomator-workspace-<id>` for every dispatched task. Its own doc comment says "workspace is only removed on success. On failure, it's preserved for debugging" — a deliberate design choice with no expiry, so failed-run workspaces accumulated forever. Found when the host hit 100% disk full (29MB free on 117GB) — 161 stale workspace dirs, ~17.6GB. Directly caused two real-time dispatch failures (generic "container execution failed: exit status 1") before being diagnosed. Manually cleaned up as an immediate workaround (161 dirs removed, ~15GB freed) before the real fix landed. + +**Why:** Preserving a failed workspace for debugging is reasonable; preserving it *forever* with no retention bound is not — the fix adds a bounded sweep rather than removing the debug-preservation behavior outright. + +**How to apply:** If a claudomator dispatch fails with a vague "exit status 1" on a host running a pre-`06c9730` binary, check `df -h /` and `du -sh /tmp/claudomator-workspace-*` before assuming a credential/infra blip — disk-full was the confirmed cause of at least 2 dispatch failures in this session alone. diff --git a/project_doot_android_roadmap.md b/project_doot_android_roadmap.md new file mode 100644 index 0000000..50613a9 --- /dev/null +++ b/project_doot_android_roadmap.md @@ -0,0 +1,16 @@ +--- +name: project-doot-android-roadmap +description: "doot Android app's native/web split and the offline-support requirement, from the 2026-08-07 architecture review" +metadata: + node_type: memory + type: project + originSessionId: 2607e018-d4ad-4ec6-bb68-5f1d0f9a3906 +--- + +The Android app is deliberately split: a native shell (Glance widget, QuickAddActivity, TaskDetailActivity, SettingsActivity) for glance/sub-second-capture surfaces, and a WebView wrapper (DashboardActivity) around the real web dashboard for everything administrative (chains, buckets, recurrence, projects, labels). The user confirmed the "glanceable" test — native only for things that must work from the home screen without opening the app, or need sub-second capture — is the right seam and endorsed keeping it as the ongoing boundary. + +**User wants offline support for the Android app.** This was stated before 2026-08-07 (not captured in memory until now — an earlier architecture-review draft treated it as a hypothetical future trigger and the user corrected that directly: "you got off gesticulating about no need for offline support when I already told you I want it"). + +**How to apply:** Don't treat offline support as optional or a someday-maybe in Android planning — it's a confirmed requirement. The architecturally consistent way to build it (matching the native/web split above): a service worker + IndexedDB write queue in the *web app itself* (caches the shell/last-fetched tab data, queues mutations like complete-atom/unified-add, replays on reconnect), not a native rewrite — this benefits a browser tab and the WebView-wrapped app identically and keeps doot's task/chain/bucket business logic living in one place (the Go server) instead of being reimplemented in Kotlin. Separately, the Glance widget talks to the bearer-token API directly over OkHttp and never touches the WebView, so a service worker does nothing for it — its own offline behavior (does a checkbox tap or quick-add fail silently when offline, or queue?) needs a small native fix of its own (a Room-backed retry queue flushed by the existing WorkManager refresh cycle), independent of the web app's offline work. + +See also [[project_doot_widget_build]] for the widget build/publish workflow. diff --git a/project_doot_google_tasks_oauth_limitation.md b/project_doot_google_tasks_oauth_limitation.md new file mode 100644 index 0000000..8657b5d --- /dev/null +++ b/project_doot_google_tasks_oauth_limitation.md @@ -0,0 +1,14 @@ +--- +name: doot-google-tasks-service-account-limitation +description: "doot's Google Tasks integration is broken by design — service account auth can't see the user's real task lists, unlike Calendar" +metadata: + node_type: memory + type: project + originSessionId: 61c70ef2-1b1b-456c-9cbd-ad1f76ea2401 +--- + +Google Tasks in doot always returns 0 tasks because `GOOGLE_CREDENTIALS_FILE` (`task-dashboard-485123-e15b31930293.json`) is a **Cloud service account** key (`doot-terst-org@task-dashboard-485123.iam.gserviceaccount.com`), not the user's personal OAuth. Confirmed by calling `GoogleTasksClient.GetTaskLists`/`GetTasks` directly against the live API: it sees exactly one list ("My Tasks") with zero items — that's the *service account's own* empty default list, not the user's. The user's actual Google Tasks account has three lists (My Tasks, Holiday prep, Computer) with multiple open items, confirmed via a screenshot of the real Tasks app. + +**Why:** Google Calendar supports sharing a calendar with any email address, including a service account's — that's why Calendar events fetch correctly through the same credentials file. Google Tasks has no equivalent list-sharing mechanism at all. A service account can never see a regular user's personal task lists, no matter how the lists are configured. This isn't a stale-ID or query-filter bug in doot's code — the current auth approach cannot work for Tasks, full stop. + +**How to apply:** Any real fix requires standard 3-legged OAuth for Google Tasks specifically (consent screen, refresh token storage) — the service-account shortcut that works for Calendar has no path forward here. Don't spend time debugging `GoogleTasksClient`/`fetchGoogleTasks`/cache logic for "why don't tasks show up" — the code is fine; the auth model is the blocker. User has explicitly deferred fixing this ("leave it for now") as of 2026-07-14. diff --git a/project_doot_isallday_midnight_bug.md b/project_doot_isallday_midnight_bug.md new file mode 100644 index 0000000..bd5bf30 --- /dev/null +++ b/project_doot_isallday_midnight_bug.md @@ -0,0 +1,22 @@ +--- +name: project-doot-isallday-midnight-bug +description: "FIXED 2026-08-10 (commit 9a3ece4): dated doot-native/Google tasks landed in the widget's undated/floating bucket (rendering as due 'today') instead of their correct day, because every task due-date is midnight-anchored and trips the IsAllDay 'is this all-day' heuristic" +metadata: + node_type: memory + type: project + originSessionId: 647d8c60-e602-440e-b8b3-d8422fcbcb1d +--- + +**Status: FIXED and deployed**, commit `9a3ece4` on 2026-08-10. Originally diagnosed 2026-07-29 and deliberately set aside; recurred as a fresh report ("tasks scheduled for tomorrow show up right on the webview but on today in the widget") which prompted implementing it. + +**Root cause:** doot-native and Google Tasks due dates are always midnight-anchored (even with a real due date), which trips `TimelineItem.ComputeDaySection`'s "midnight means no specific time" heuristic and sets `IsAllDay = true` on them. `TimelineItemToWidgetItem` then left `wi.Start` nil for any Task/GTask with `IsAllDay == true`, and the Android client's undated/floating pool (`DootWidget.kt`'s `floating` list) has zero per-day awareness — it just packs items forward from "now" — so a task due tomorrow rendered as if due today. + +**Important correction to the original 2026-07-29 fix plan:** that plan proposed gating `ComputeDaySection`'s midnight heuristic to `Event`/`Meal` types only. Re-verifying it against the live code (rather than implementing on trust) surfaced a real problem: it would also flip `IsAllDay` to `false` for *same-day* dated tasks, moving them on the **web** from the untimed-item strip into the hourly calendar grid with a fabricated "12:00 AM" time label — fixing the widget by breaking the web's already-correct rendering. `IsAllDay`/`ComputeDaySection` is shared by both clients; the web buckets by `DaySection` (computed from `Time`, unaffected by `IsAllDay`) and only uses `IsAllDay` for the same-day untimed-vs-grid visual choice, which the naive fix would have broken. + +**Actual fix implemented:** added `TimelineItem.Undated bool` — a signal genuinely independent of `IsAllDay`, true only for the caller's two genuinely-dateless constructions in `internal/handlers/timeline_logic.go` (nativeUndated tasks, gtasks with no due date), both of which use `Time = now` as a layout placeholder rather than a real date. `ComputeDaySection`/`IsAllDay` were left completely untouched (zero web-side risk). `TimelineItemToWidgetItem` routes on `Undated` instead: dated Task/GTask items now get `wi.Start` populated (so the Android client's existing, already-correct Start-based day bucketing — `todayScheduledEvents`/`tomorrowItems` in `DootWidget.kt` — runs for them), while genuinely undated ones keep the original nil-Start floating treatment. + +Also fixed the identical latent bug in `wi.DueDate`'s guard in the same pass: it only checked `!Time.IsZero()`, but `nativeUndated`'s `Time=now` placeholder is non-zero, so an undated task's Android detail popup would show a fabricated due date. Now guards on `!Undated` too. + +**Verification:** new tests `TestTimelineItemToWidgetItem_DatedGTaskGetsStart` and `_UndatedTaskWithPlaceholderTime_NilDueDate`, plus three existing tests updated because they had encoded the bug's "Start stays nil for any IsAllDay task" as expected behavior. Proved the fix by reverting `widget.go` to the pre-fix condition against a real backup and confirming exactly the three tests exercising the new behavior failed, then restored and confirmed byte-identical. `go build`/`vet`/`test` all clean; deployed to tungsten and health-checked. + +**Takeaway for future similar reports:** when a proposed fix and its own diagnosis are more than a session old, re-derive the trace against current code before implementing rather than trusting the old plan verbatim — the plan itself had a real, would-have-shipped bug (web-side regression) that only surfaced from re-checking, not from the original diagnosis. diff --git a/project_doot_prod.md b/project_doot_prod.md new file mode 100644 index 0000000..33e00a4 --- /dev/null +++ b/project_doot_prod.md @@ -0,0 +1,48 @@ +--- +name: project-doot-prod +description: "Production deployment facts for doot.terst.org — deploy target is tungsten ONLY (migrated from titanium 2026-07-17); verified architecture facts as of the first real deploy on 2026-07-17" +metadata: + node_type: memory + type: project + originSessionId: 31afbee3-b8e9-4163-a1bf-ea5392aef019 +--- + +**Update (2026-07-17): production migrated from titanium to tungsten.** The deploy host is now **tungsten** — the user confirmed this explicitly ("we migrated to tungsten so that is now the deploy host"). This same day, the first real deploy since the migration ran successfully from a tungsten sandbox and corrected several stale facts below (previously copied from a box that only *resembled* prod — see the cautionary tale). Everything in this file is now verified against that real deploy, not inferred from resemblance. + +**Verification: run `hostname`.** Reports `tungsten` on the deploy host. Always check before any deploy-shaped action; if it's not `tungsten`, do the build/test/commit/push work and stop there, or ask. `scripts/deploy` (see below) now enforces this itself and refuses to run anywhere else — but still verify independently before invoking it, since the classifier gating deploy-shaped Bash actions wants fresh, visible confirmation in the same turn, not a memory citation. + +**Cautionary tale (pre-migration, still the reason for the rule):** In one session, "deploy = build + restart" was performed on a sandbox that wasn't the deploy host: built the Go binary in place, ran DB migrations against its `dashboard.db`, restarted the service — believing this was production. It wasn't; had to be unwound (stop/disable service, restore DB from backup, delete the built binary). Lesson: **confirm the current host is the deploy host before any deploy-shaped action — don't infer it from environment resemblance.** + +--- + +**Real deploy mechanism, confirmed 2026-07-17:** run `scripts/deploy` from the repo root (e.g. `/workspace/doot`). It now has a `DEPLOY_HOST="tungsten"` guard at the top that hard-exits if `hostname` doesn't match — added after that session's classifier explicitly gated the raw deploy steps and asked for this kind of self-enforcement. The script: builds CSS (`npm run css:build`), builds the Go binary with `-ldflags` version stamping, stops the service, rsyncs `web/static`→public, `web/templates`, and `migrations` into the site dir, moves the binary in, fixes ownership/perms, restarts the service, and prints `systemctl status`. + +**Two other "deploy" scripts existed and were deleted 2026-07-17 as abandoned scaffolding:** `deployment/deploy` (a near-duplicate of `scripts/deploy`) and `deployment/post-receive` (a git post-receive hook meant to be installed at `/site/doot.terst.org/app-code/hooks/post-receive` for a push-to-deploy workflow) — confirmed the hook was never actually installed (no `.git/hooks/post-receive` existed) and the `app-code` checkout it depended on was stale (some files untouched since Jan, well predating current work). That checkout has since been `rm -rf`'d. `deployment/apache.conf` and `deployment/task-dashboard.service` are still sitting in the repo — known stale/needs-a-new-home per the user, but no destination decided yet, so left in place. + +**Verified real paths (previous versions of this memory had these wrong — they were guessed from a resembling sandbox, not checked):** +- Binary: `/site/doot.terst.org/app` (NOT `/workspace/doot/app` — that's just the source checkout the deploy script builds from before moving the binary over) +- DB: `/site/doot.terst.org/data/dashboard.db` (NOT `/workspace/doot/dashboard.db`, and NOT `/site/doot.terst.org/dashboard.db` either — that top-level path is a stale 0-byte decoy file that's easy to query by mistake; the real one is one level deeper, in `data/`. Confirmed via `DATABASE_PATH` in `/site/doot.terst.org/.env`.) +- Env: `/site/doot.terst.org/.env` +- Service: **`task-dashboard@doot.terst.org.service`** (a templated unit, `/etc/systemd/system/task-dashboard@.service`) — NOT `doot.service`. `systemctl restart task-dashboard@doot.terst.org.service` / `journalctl -u task-dashboard@doot.terst.org.service`. +- Static/templates/migrations synced into `/site/doot.terst.org/public`, `/site/doot.terst.org/templates`, `/site/doot.terst.org/migrations` respectively by the deploy script. +- DEFAULT_PASS in .env is a placeholder (user already exists in DB; EnsureDefaultUser only runs if users table is empty) + +**Fresh-sandbox gotcha: `node_modules` can look installed but be broken.** `npm install` reported "up to date, audited 107 packages" while the actual `postcss` CLI binary was non-functional (`Cannot find module '.../node_modules/.bin/lib/args.js'`), because `node_modules` only had 98 stale entries and was missing `postcss-cli` entirely. Fix: `rm -rf node_modules && npm ci` (there's a `package-lock.json`) before trusting `npm run css:build` / `scripts/deploy`. + +**Migration system tracks by full filename, not numeric prefix** (`internal/store/sqlite.go` `runMigrations`, table `schema_migrations`). See [[feedback-migration-renumbering]] for the incident this caused (a renamed already-applied migration crash-looped the service after this deploy) and the rule going forward (never rename/renumber an existing migration file to resolve a merge collision). + +**Git config:** `git config --global --add safe.directory /site/git.terst.org/repos/doot.git` has already been granted and applied (global, persists across sessions on this box) — pushes to the `local` remote from a root shell won't hit the "dubious ownership" error again for this specific path. A similar error will still occur for other www-data-owned paths (e.g. it happened for the now-deleted `/site/doot.terst.org/app-code`) until the same exception is added for them too, and that always needs to be asked for explicitly first (never run `git config --global` unprompted). + +**Reverse proxy: Apache (NOT Caddy) — this memory previously had it backwards.** Corrected 2026-08-04 after checking live: `caddy.service`/`caddy-api.service` are both `disabled` and not running; Apache2 is what's actually listening on :80/:443 (confirmed via `ss -tlnp` and `apache2ctl -S`). The `doot/deployment/apache.conf` mentioned above as "stale/unused" was WRONG — it's real config, just not the one checked into the repo (live vhosts are `/etc/apache2/sites-available/doot.terst.org.conf` and `doot.terst.org-le-ssl.conf`, proxying `/` to `http://127.0.0.1:38080/` — the app's actual port per `PORT=38080` in `.env`, not 8080 as previously assumed here). Apache's `ProxyPass` config is separate from any TLS cert renewal mechanism, so also don't assume Caddy handles ACME either — check `/etc/letsencrypt/` (certbot) vs Caddy's automatic HTTPS before trusting either claim. This was learned the hard way: don't trust old infra claims in memory without a live check when debugging anything proxy/reachability related. See [[feedback-verify-before-asserting-root-cause]] for the fuller incident writeup this correction came out of. +- stash.terst.org → localhost:8000 (unverified whether this one's still Caddy or also moved to Apache — recheck before relying on it) + +**Playground web demo:** +- /workspace/playground/web/server.py (stdlib Python, port 9090) +- Service: /etc/systemd/system/playground-web.service +- Exposed via doot gateway at /playground/ (PLAYGROUND_URL=http://127.0.0.1:9090 in doot .env) + +**Other services:** +- stash: /workspace/stash (FastAPI/uvicorn, port 8000) — separate postgres on 5432 +- Claudomator: not currently running; would be on port 8484 + +**Git remotes:** `github` (github.com:thepeterstone/doot.git) works from any sandbox. `local` (/site/git.terst.org/repos/doot.git) is a local filesystem path that only exists on the deploy host itself (tungsten) — its absence on a given box is itself a signal that box is NOT the deploy host. diff --git a/project_doot_widget_build.md b/project_doot_widget_build.md new file mode 100644 index 0000000..6498dfa --- /dev/null +++ b/project_doot_widget_build.md @@ -0,0 +1,33 @@ +--- +name: project-doot-widget-build +description: "How to build and publish the doot Android widget APK from a fresh sandbox (toolchain bootstrap, signing choice, publish location)" +metadata: + node_type: memory + type: project + originSessionId: f8b83784-e0ed-462c-8310-7de6aadc9650 + modified: 2026-08-15T18:05:20.824Z +--- + +**Always `git fetch github && git merge --ff-only github/master` before building.** First time this was done, the sandbox's checkout was 3 days stale and missing 90 commits (a full widget refactor plus recurrence/labels-projects/budgets/multi-day-events/text-size features) — the APK got built and published from that stale tree with none of it. Don't trust `git status`'s "ahead of X by N commits" framing without checking which remote — the `local` remote (`/site/git.terst.org/repos/doot.git`) hits a "dubious ownership" error in sandboxes and its stale tracking ref can make the branch look ahead when it's actually just behind `github/master`. Check divergence against `github/master` specifically (`git log --oneline master..github/master`), and don't run `git config --global --add safe.directory` to silence the `local` remote error without asking first — fixing it isn't necessary since `github` is the remote that matters for this workflow. + +**Build environment is not preinstalled in a fresh sandbox** — bootstrap before building: +- Gradle: `android/gradlew` prefers a system Gradle at `/opt/gradle/gradle-8.6/bin/gradle` and only falls back to the repo's embedded wrapper jar if that's missing. The fallback path is broken in this environment (`NoClassDefFoundError: org/gradle/wrapper/IDownload`), so always install Gradle 8.6 to that exact path first: download `https://services.gradle.org/distributions/gradle-8.6-bin.zip`, unzip to `/opt/gradle/`. +- Android SDK: install to `/opt/android-sdk` (matches `android/local.properties` → `sdk.dir=/opt/android-sdk`). Use the cmdline-tools zip (`commandlinetools-linux-*_latest.zip` from `dl.google.com/android/repository/`), place under `/opt/android-sdk/cmdline-tools/latest/`, then `sdkmanager --sdk_root=/opt/android-sdk "platform-tools" "platforms;android-34" "build-tools;34.0.0"` (accept licenses with `yes |`). As of 2026-08-06 also installed `"build-tools;36.0.0"` and `"platforms;android-36"` / `"system-images;android-36;google_apis;x86_64"` — see the target-device note below for why. + +**Target device is Android 16 (API 36) — minSdk=targetSdk=compileSdk=36 is correct, not a bug to "fix" toward broader compatibility.** This is a single-user app sideloaded onto exactly one phone, never distributed broadly, so "what does the widest range of devices support" is the wrong framing entirely — corrected 2026-08-06 after a prior pass (2026-08-05) wrongly walked minSdk back to 31 assuming "most real devices" needed supporting, without checking what device this actually runs on. Don't re-apply that "fix" — ask or check first if a future session is tempted to lower these again. AGP 8.2.0's bundled D8 prints `API level of 36 is not supported by this compiler` at compileSdk/targetSdk=36 regardless of build-tools;36.0.0 being installed (AGP doesn't delegate to the standalone SDK build-tools binary) — this is harmless noise, not a real problem: verified by actually installing and launching the resulting APK on a real API 36 target (see below), confirmed no crash, `target_sdk_version=36` correctly applied. A genuine fix would mean upgrading AGP itself, a bigger and riskier change not currently justified. + +**APK file size jump (11MB -> 27MB) is from minSdk=36 alone, NOT code bloat -- don't re-diagnose this as a Compose/Glance/dependency problem.** Isolated and verified 2026-08-06 by rebuilding the pre-minSdk-36 commit (0400e59) in a throwaway git worktree, changing ONLY minSdk 26->36 with zero other changes, and diffing the packaged APK: `classes.dex` goes from 63%-compressed (Deflate) to 0% (Stored) the moment minSdk crosses whatever threshold AGP uses to auto-switch to uncompressed dex packaging (ART on modern API levels can mmap dex directly from the APK without a decompression step at install time -- larger file, faster/lower-memory install, a deliberate platform tradeoff). An earlier same-day attempt to explain this via `apkanalyzer dex packages` (measuring Compose/Glance/DataStore's share of the dex class tree) was a dead end -- that metric stayed ~11MB in both the small and large builds, because it measures reachable-class-tree size, not the packaged/compressed byte size that actually determines the APK file size. If asked again "what got bigger," check compression method via `unzip -v <apk> classes.dex` (Cmpr column) before reaching for a dependency-based explanation. + +**Dedicated API 36 test AVD: `doot_test_api36`** (`system-images;android-36;google_apis;x86_64`, created 2026-08-06, separate from [[project-feedreader-emulator|feedreader_test]] -- don't reuse that one for doot, it's a different project's dedicated resource). Boot with `emulator -avd doot_test_api36 -no-window -no-audio -no-boot-anim`, then poll both `getprop sys.boot_completed` AND `getprop dev.bootcomplete` (both must be `1`) before `adb install` -- installing too early fails with `Can't find service: package` or `Error: device is still booting.` even after `sys.boot_completed=1` alone. Use this AVD to verify installability/launch after any build.gradle.kts SDK-version change, the same way the minSdk=36→31 mistake was originally caught: don't just declare a target API and assume it works, actually install onto a real matching target and confirm no crash via `logcat | grep FATAL`. + +**Build debug, not release:** +- `android/app/build.gradle.kts` has a `release` signingConfig pointing at `android/doot-widget.keystore` — that file is gitignored and not present in any sandbox seen so far (only exists wherever the original release key was generated, possibly the deploy host or a personal machine — never found it). +- Even if it were available, **prefer the debug build anyway**: sideloaded release-signed APKs from a low-reputation self-signed key get flagged hard by Google Play Protect ("app blocked as harmful"). Sibling projects [[project-doot-prod|can-can and nav]] avoid this by never wiring a release signingConfig at all and just shipping debug-signed APKs for sideloading (nav even has a checked-in `app-debug.apk`). Matched that pattern for doot's widget. +- Command: `cd /workspace/doot/android && export ANDROID_HOME=/opt/android-sdk && ./gradlew :app:assembleDebug` +- Output: `android/app/build/outputs/apk/debug/app-debug.apk` + +**Publish location:** copy the built APK to `/site/static.terst.org/public/files/doot-widget.apk`, then `chown www-data:www-data` it. A `doot-widget.apk` already existed there from a prior session (this is the established convention/URL the user installs from), so this is an overwrite-in-place, not a new path. + +**How to apply:** This publish step is a plain static-file write, not a service restart or DB migration — it does **not** fall under the "only deploy on tungsten" rule in [[project-doot-prod]] (deploy host migrated from titanium to tungsten 2026-07-17). That rule is specifically about the doot *server* (systemctl restart, dashboard.db migrations). Publishing the widget APK to the static file server is safe to do from any sandbox as long as `/site/static.terst.org/public/files/` is reachable. + +**If the build OOM-kills on a RAM-constrained host** (hit on titanium, 3.8GB total RAM, 2026-07-16, building `assembleRelease`): the persistent gradle daemon + kotlin compile daemon each reserve ~512MB-1GB, and a stale daemon left over from a prior failed build compounds it. Check `free -h` first; if tight, `./gradlew --stop` and `pkill -f KotlinCompileDaemon` to reclaim memory, then build with `--no-daemon --max-workers=1 -Dorg.gradle.jvmargs="-Xmx1024m -XX:MaxMetaspaceSize=256m"` appended to whichever assemble command you're running. Not hit on tungsten as of 2026-08-14 (debug builds succeeded with no special flags) — this is a fallback for a constrained host, not the default recipe. diff --git a/project_feedreader.md b/project_feedreader.md new file mode 100644 index 0000000..d884ee4 --- /dev/null +++ b/project_feedreader.md @@ -0,0 +1,25 @@ +--- +name: project-feedreader +description: "FeeDreader (formerly \"Feeds\") Android RSS reader app (org.terst.feedreader) built for Peter — standalone greenfield project, separate from doot" +metadata: + node_type: memory + type: project + originSessionId: 375c858e-e817-41e8-a076-7920d22eab2b +--- + +Building "FeeDreader" (renamed 2026-07-23 from "Feeds" — pun on "fee dreader", avoiding +paywalls), a standalone Android RSS/Atom reader app for Peter, package `org.terst.feedreader`, +repo at `/workspace/feedreader` (separate git repo from doot, same local git identity). +Greenfield build authorized with "just go for it - greenfield, simple design, just build." + +Core feature set: RSS/Atom subscriptions (now with OPML-category folder grouping + +case-insensitive sort, and a virtual "Last fetch failed" section), full article + +WebView/Readability.js reader-mode view, OPML import/export, keyword mute/highlight filters, +save-to-Stash integration (explicit action, not automatic), Android share-sheet target. +Explicitly excluded: local full-text search, home-screen widget, notifications, social +features. Has an owl-themed adaptive launcher icon (hand-authored vector, no external design +tool available) and a short paywall-avoidance philosophy note in Settings. + +**Why:** Inoreader (his prior RSS reader) removed OPML export, forcing a migration off it — his ~107 feeds were pulled via Inoreader's Reader API (ClientLogin) and re-packaged as OPML for import into this app. + +**How to apply:** Peter's workflow for this app is hands-on testing rounds — he sideloads a build to his phone, uses it for a while, then sends a batch of bug reports/feature requests in one message. Expect iterative APK rebuild-and-resend cycles, not a single big spec upfront. Room DB schema is at v6 as of 2026-07-23 (added `FeedEntity.folder`); see `AppDatabase.kt`'s migration comments — several past migrations needed a full table rebuild (Room validates exact column match, not just superset) rather than a plain ADD COLUMN. See [[feedback_feedreader_apk_naming]] and [[project_feedreader_emulator]]. diff --git a/project_feedreader_emulator.md b/project_feedreader_emulator.md new file mode 100644 index 0000000..6b11e78 --- /dev/null +++ b/project_feedreader_emulator.md @@ -0,0 +1,26 @@ +--- +name: project-feedreader-emulator +description: "Android emulator set up on tungsten for direct testing of the feedreader (\"Feeds\") app instead of relying only on Peter's phone" +metadata: + node_type: memory + type: project + originSessionId: 375c858e-e817-41e8-a076-7920d22eab2b +--- + +An Android emulator is provisioned on tungsten (the doot/feedreader host) for testing the "Feeds" app directly: AVD name `feedreader_test`, API 34, google_apis, x86_64, pixel_6 profile. Launch headless: `-no-window -no-audio -no-boot-anim -gpu swiftshader_indirect`. KVM-accelerated (host has `/dev/kvm` + AMD SVM). Controlled via `adb shell input tap/swipe`, `adb shell screencap`, and `adb shell uiautomator dump` for exact element bounds (more reliable than estimating tap coordinates from screenshots). + +**Why:** Peter suggested this ("since you're on tungsten you should have enough resources to install an emulator") after a hard-to-reproduce "crash loop" bug report — sideload-to-phone-and-wait-for-feedback cycles were too slow for debugging. With the emulator, a real crash (WebView touched off the main thread during Readability-extraction cancellation) was reproduced directly via scripted rapid-swiping through articles, root-caused from an actual stack trace, and verified fixed under a heavier stress test — instead of guessing from code review alone. + +**How to apply:** For future feedreader bugs, prefer reproducing on this emulator (install APK, drive via adb, capture logcat) before making a fix, rather than shipping a guess and waiting on Peter's phone-based feedback. `adb exec-out run-as org.terst.feedreader sqlite3 /data/data/org.terst.feedreader/databases/feedreader.db "..."` can query the on-device Room DB directly to verify behavior (e.g. confirming isRead flips). The SDK at `/opt/android-sdk` isn't on PATH by default — export `PATH=$PATH:/opt/android-sdk/platform-tools:/opt/android-sdk/emulator` first, and check `adb devices` since the emulator is often already running. + +To test `ReaderExtractor` against an arbitrary URL not in a subscribed feed, fire a share +intent at `ShareReceiverActivity` rather than adding a throwaway subscription: +`adb shell am start -a android.intent.action.SEND -t text/plain --es android.intent.extra.TEXT "<url>" -n org.terst.feedreader/.ui.ShareReceiverActivity` +— it extracts and saves to the Saved list (isSaved=true), viewable afterward. + +**Caveat confirmed 2026-07-23:** extracting a heavy, ad/tracker-laden real-world page (e.g. an +Ars Technica article) through this emulator is genuinely flaky — the same URL can time out, +succeed fully, or succeed with less content (some page sections render in via client-side JS +some time after `onPageFinished`, racing against when our extraction script runs) across +back-to-back identical attempts with zero code changes in between. Retry 2-3 times (waiting +the full ~20-30s) before concluding a code change broke extraction. diff --git a/project_marine_cert_prep.md b/project_marine_cert_prep.md new file mode 100644 index 0000000..af3ca16 --- /dev/null +++ b/project_marine_cert_prep.md @@ -0,0 +1,16 @@ +--- +name: project-marine-cert-prep +description: "What the user's \"Marine Cert Prep\" doot chain is for — NMEA MEI certification study plan" +metadata: + node_type: memory + type: project + originSessionId: 119bd1a5-fb32-4222-9695-fcbe2c8fd63b +--- + +The user is working toward NMEA Marine Electrical Installer (MEI) certification. Tracked as a 16-step locked/linear task chain in doot (see [[reference_doot_marine_codemap]] for exact IDs/DB/code locations). + +Created 2026-07-17. As of creation, only step 0 ("Pass NMEA DC Electrical Overview online prerequisite") is unlocked/due; the rest unlock one at a time on completion (WIP-1 chain). + +Plan covers, in order: the NMEA online prerequisite, ABYC E-11 DC electrical rules (7-40-72 rule, battery switch placement, AIC parameters, temperature/bundling de-rating, voltage-drop budgets), a wire-sizing cheat sheet, AC/DC grounding and ELCI/galvanic-isolation rules, a distance/dimension cheat sheet, the ABYC Marine Electrical practice exams (target >85% before booking the proctored exam), an FCC Amateur Radio General license (via HamStudy.org, focused on G5/G6/G9), and finally a logged hands-on volunteer electrical audit at Honokohau to start the 1-year NMEA experience clock. + +**How to apply:** treat this as a personal certification study project, not a software project — "code" work here means reading/updating doot's chain data (via DB or the chain API/UI), not writing new features, unless the user separately asks for a doot code change. diff --git a/reference_doot_marine_codemap.md b/reference_doot_marine_codemap.md new file mode 100644 index 0000000..4d46fe1 --- /dev/null +++ b/reference_doot_marine_codemap.md @@ -0,0 +1,31 @@ +--- +name: reference-doot-marine-codemap +description: "Code map for locating the \"Marine\" project (Marine Cert Prep task chain) in doot's DB and Go source" +metadata: + node_type: memory + type: reference + originSessionId: 119bd1a5-fb32-4222-9695-fcbe2c8fd63b +--- + +The user's "marine project" is a **doot task chain** named "Marine Cert Prep" — NOT a separate repo. It lives entirely inside the [[project_doot_prod]] `doot` app's data and code. + +## Identifiers +- `project_id = 410a417491154b18ee3ff9cb` (row in `projects` table, name "Marine Cert Prep", color `#8B5CF6`) +- `chain_id = 4ad4ec5d4b133658fb3e0ac0` (row in `task_chains` table, status `active`), also cached at `/tmp/marine_chain_id.txt` +- 16 sequential `native_tasks` rows (`chain_position` 0-15) with `chain_id` set to the above — a linear WIP-1 study plan for the NMEA Marine Electrical Installer (MEI) certification, covering ABYC E-11 standards, DC electrical calcs, an FCC Amateur Radio General exam, and a hands-on audit log. + +## DB access +- Production DB: `/site/doot.terst.org/data/dashboard.db` (SQLite, owned by `www-data`). WAL files sit alongside it; there's also a `.bak-<timestamp>` snapshot from the last migration. +- No `sqlite3` CLI installed on this box — query via `python3 -c "import sqlite3; ..."` instead. +- Relevant tables: `projects`, `task_chains`, `native_tasks` (has `chain_id`/`chain_position`/`chain_unlocked` columns, added in `migrations/026_task_chains.sql`). +- Schema source of truth: `/workspace/doot/migrations/026_task_chains.sql` (chains) and `020_native_tasks.sql` (base table). + +## Go code (repo: `/workspace/doot`) +- Store layer: `internal/store/chains.go` — `CreateChain`, `GetChain(id)`, `GetChains()`, `GetChainTasks(chainID)`, `SetChainStatus(id, status)`, `advanceChain` (unlock-next-on-complete logic). +- Task completion hook that advances chains: `CompleteNativeTask` in `internal/store/native_tasks.go` (returns `ErrChainTaskLocked` if you try to complete a locked step). +- Web handlers: `internal/handlers/chains_web.go` — `BuildChainSummaries`, `HandleChainDetailView`, `HandleChainPause/Resume/Abandon`. +- Routes (`cmd/dashboard/main.go` ~L330-414): web `GET/POST /chains/{id}[/pause|resume|abandon]`; widget API `POST /api/widget/chains`, `GET /api/widget/chains/{id}`, `POST /api/widget/chains/{id}/{pause,resume,abandon}`. +- Agent Context API (external read/write access, bearer-token gated): `internal/handlers/agent.go` — `HandleAgentContext` (GET `/agent/context`), task write/complete endpoints. Auth flow: `POST /agent/auth/request` → browser approval → `GET /agent/auth/poll`. + +## How to apply +When the user says "my marine project," resolve it to `project_id 410a417491154b18ee3ff9cb` / `chain_id 4ad4ec5d4b133658fb3e0ac0` and go straight to the DB/code above rather than searching for a "marine" repo or directory — there isn't one. [[feedback_deploy_classifier]] and [[feedback_pull_before_build]] still apply if any change here touches the live doot deploy. diff --git a/reference_google_tasks_due_date_utc_quirk.md b/reference_google_tasks_due_date_utc_quirk.md new file mode 100644 index 0000000..a0025dc --- /dev/null +++ b/reference_google_tasks_due_date_utc_quirk.md @@ -0,0 +1,16 @@ +--- +name: reference-google-tasks-due-date-utc-quirk +description: "Google Tasks API's due date field is date-only but always serialized as midnight UTC -- converting the instant into a display timezone (vs. re-anchoring the date) shifts dates in negative-offset zones" +metadata: + node_type: memory + type: reference + originSessionId: 647d8c60-e602-440e-b8b3-d8422fcbcb1d +--- + +The Google Tasks API's `due` field is conceptually date-only (no time-of-day is meaningful), but the API always serializes it as an RFC3339 timestamp fixed at midnight UTC, regardless of the user's actual timezone. E.g. a task due "July 29" comes back as `"2026-07-29T00:00:00.000Z"` even for a user in Honolulu. + +**The bug this causes:** if code does `dueDate.In(displayTZ)` on that parsed instant (converting the moment-in-time into the display timezone) instead of re-anchoring the same Y/M/D to local midnight, the date silently shifts backward for any timezone behind UTC. Midnight UTC becomes 2pm the *previous* day in UTC-10 (Pacific/Honolulu, doot's configured `config.GetDisplayTimezone()`) -- so every dated Google Task landed one calendar day earlier than Google actually reported. Symptom: "I have a scheduled task for tomorrow that's showing up today." + +**The fix** (`internal/api/google_tasks.go`, `getTasksFromList`): extract `y, m, d := dueDate.Date()` from the UTC-parsed timestamp (that Y/M/D *is* the intended calendar date) and rebuild `time.Date(y, m, d, 0, 0, 0, 0, displayTZ)` from those components -- never call `.In(displayTZ)` on a Google Tasks due-date instant directly. Fixed 2026-07-28, regression-tested in `google_tasks_test.go` (`TestGetTasksFromList_DueDateNotShiftedByNegativeOffsetTZ`) using the real Pacific/Honolulu location via a mocked HTTP tasks-list response. + +**How to apply:** if any *other* date-only field from an external API gets added to doot (a new integration, or a different Google Tasks/Calendar field), check whether that field has the same "date-only-but-serialized-as-UTC-midnight" convention before doing a naive `.In(displayTZ)` conversion on it. This is a genuinely common API convention (RFC3339 doesn't have a "date-only" type), not unique to Google Tasks. diff --git a/reference_hawaii_gis_sources.md b/reference_hawaii_gis_sources.md new file mode 100644 index 0000000..df55da8 --- /dev/null +++ b/reference_hawaii_gis_sources.md @@ -0,0 +1,106 @@ +--- +name: hawaii-gis-sources +description: Verified ArcGIS REST API endpoints for Hawaii County public data dashboard at /workspace/hawaii +metadata: + node_type: memory + type: reference + originSessionId: 9b72719b-f22b-47bb-aabc-ac5449f58b43 +--- + +Two GIS servers with queryable public layers. All use ArcGIS REST API pattern: +`/MapServer/{layerID}/query?geometry=...&geometryType=esriGeometryEnvelope&inSR=4326&outSR=4326&outFields=*&f=geojson` + +## Server 1: geodata.hawaii.gov (State GIS) +Base: `https://geodata.hawaii.gov/arcgis/rest/services/{Service}/MapServer/{ID}` + +### Hazards +| Layer | Endpoint | Notes | +|---|---|---| +| Lava Flow Hazard Zones | Hazards/MapServer/3 | Hawaii County ONLY. hzone 1-9 (1=highest near vents) | +| Tsunami Evacuation Zones | Hazards/MapServer/2 | Statewide incl. Hawaii Island. Updated Nov 2025 | +| Hawaii County DFIRM (Flood) | Hazards/MapServer/14 | FEMA flood zones (AE, V, X etc). Updated Dec 2025 | +| Fire Risk Areas | Hazards/MapServer/7 | risk field: High/Medium/Low | +| Coastal Flood w/ 3.2ft SLR | Hazards/MapServer/21 | Hawaii Island only | + +### Infrastructure +| Layer | Endpoint | Notes | +|---|---|---| +| Public Schools | Infrastructure/MapServer/7 | Filter island=Hawaii | +| County Parks (Hawaii Co.) | Infrastructure/MapServer/17 | Hawaii County ONLY | +| Hospitals | Infrastructure/MapServer/5 | Statewide | +| Wastewater Treatment Plants | Infrastructure/MapServer/29 | Statewide | +| Onsite Sewage (Hawaii Island) | Infrastructure/MapServer/26 | Hawaii County ONLY | + +### Emergency +| Layer | Endpoint | +|---|---| +| Fire Stations | EmergMgmtPubSafety/MapServer/7 | +| EMS Stations | EmergMgmtPubSafety/MapServer/8 | +| Police Stations (Hawaii Island) | EmergMgmtPubSafety/MapServer/12 | + +### Land Use +| Layer | Endpoint | Notes | +|---|---|---| +| TMK Parcels (current) | ParcelsZoning/MapServer/5 | In use already | +| County Zoning | ParcelsZoning/MapServer/2 | Hawaii County ONLY | +| LUPAG | ParcelsZoning/MapServer/22 | Land Use Pattern Allocation Guide | +| Development Plan Areas | ParcelsZoning/MapServer/24 | Community plan boundaries | + +### Transportation +- Hawaii County Roads: Transportation/MapServer/6 (FULLNAME, CLASS, DISTRICT) +- Hawaii County Major Roads: Transportation/MapServer/8 + +### Climate/Sea Level Rise +- Sea Level Rise 3.2ft: Climate/MapServer (31 sub-layers at IDs 40+) +- Rain gauges, solar, wind data also available + +### Cultural/Historical +- Ahupuaa: HistoricCultural/MapServer/1 (traditional Hawaiian land divisions) +- Moku: HistoricCultural/MapServer/3 +- Na Ala Hele Trails: Terrestrial/MapServer/34 + +--- + +## Server 2: gis.hawaiicounty.gov (County GIS) +Base: `https://gis.hawaiicounty.gov/arcgis/rest/services/` + +### Planning Permits (all queryable, up to 5000 records) +Service: `Planning_Department_GIS_Public_Layers/MapServer` +- SMA Major Permits: Layer 93-94 (APPLICANT, HEARING_DA, TMK, STATUS) +- SMA Minor Permits: Layer 95-96 +- Use Permits: Layer 99-100 +- Variances: Layer 101-102 +- Ohana Permits: Layer 91-92 +- Special Permits: Layer 97-98 +- County Zoning (detailed): Layer 128 (ZONE, ORDINANCE, DATE_) + +### Wastewater/Sewer Infrastructure +Service: `Wastewater/Wastewater_Division_Public_Data/MapServer` +- Sewer Mains (polylines): Layer 3 +- Manholes: Layer 2 +- Treatment Plants & Pump Stations: Layer 5 +- Cesspools: Layer 8 +- Large Capacity Cesspools: Layer 7 + +### Traffic +- Traffic Signals: `Public_Works_Traffic/Traffic_Division_GIS_Signals_Public/MapServer/0` +- Street Lights: `Public_Works_Traffic/Traffic_Division_GIS_Street_Lights_Public/MapServer/0` + +### Geocoding +- Address search: `DIT/County_of_Hawaii_Address_Search_Service` (GeocodeServer) +- Parcel search by TMK: `DIT/County_of_Hawaii_Parcel_Search_Service` (GeocodeServer) + +--- + +## Building Permits — Key Limitation +Hawaii County uses Tyler Technologies EnerGov (EPIC system). **No public REST API for bulk building permit data.** +Planning permits (SMA, use, variance) ARE queryable via gis.hawaiicounty.gov above. + +## Token-Gated (not publicly accessible) +- Civil Defense data +- Real Property Tax (RPT) division data +- Long Range Planning (LRP) +- OHCD (housing) +- Broadband coverage layer exists but token-required +- Hele-On bus routes/stops (GTFS may be available separately) +- Water Supply DWS (dashboard only, no exposed feature service) |