docs: add OSS replacement redesign plan

Captures the locked architectural decisions from the chatbot-driven
redesign discussion: SQLite as source of truth with external trackers
as input feeds, events table replacing ad-hoc task columns, per-runner
structured agent channel, three-runner roster with classifier, scoped
auth, multi-dimensional budget, per-runner sovereign resume.

https://claude.ai/code/session_01SESwn7kQ7oP62trWw6pc39

1 files changed, 199 insertions, 0 deletions

diff --git a/docs/plans/claudomator-oss-replacement.md b/docs/plans/claudomator-oss-replacement.md
new file mode 100644
index 0000000..f0fd3dc
--- /dev/null
+++ b/docs/plans/claudomator-oss-replacement.md
@@ -0,0 +1,199 @@
+# Claudomator OSS Replacement — Architectural Redesign
+
+## Context
+
+Claudomator today is a useful but overgrown harness: ~10 task states, a stories layer, an elaborate-via-REST endpoint, file-based agent escape hatches, a manual task-creation web form, and a stub GeminiRunner that silently returns canned output. The primary mode of interaction we actually want is now: **a chatbot drives task creation and orchestration on the user's behalf; the human only intervenes when an agent asks a question; the web UI is for observability, not for primary input.**
+
+This plan locks the architectural shape of that redesign. Implementation phasing is sketched at the end but the focus here is decision capture so subsequent work can proceed without re-litigating.
+
+**Mission:**
+- Reduce custom code we maintain.
+- Support chatbot-as-primary-driver (us, or any chatbot with MCP).
+- Push-notify the user when human input is needed.
+- Web UI = observability layer (task tree, live logs, event stream, accept/reject).
+- Multi-runner (Claude cloud, Gemini cloud, OSS local) with budget-aware routing.
+
+**Non-goals (deferred):**
+- Cross-model resume — each runner is sovereign over its own continuity.
+- Non-code-task UX optimization — shape must remain *possible* (project_dir is optional) but no code or docs accommodate it yet.
+- Autonomous chatbot responding to agent questions without human — the subscription model must not foreclose it, but it's not built.
+
+---
+
+## Locked Decisions
+
+### 1. SQLite is source of truth; external trackers are input feeds
+
+Tasks live in our `tasks` table. GitHub/GitLab/whatever-tracker integrations create and update tasks through the existing webhook surface — they are not the data model. Each task carries an `external_refs` slice (`[{tracker, url}]`) for opportunistic back-syncing of state, but nothing load-bearing depends on the external tracker being available.
+
+Rejected: GitHub Issues as source of truth (Option B). Too much lock-in, non-code tasks fit poorly, and we'd still need our own DB for execution data.
+
+### 2. Events table replaces ad-hoc task-state columns
+
+A single `events` table replaces `question_json`, `interactions_json`, `summary`, `elaboration_input` on `tasks`. Schema:
+
+```
+events: id, task_id, seq, ts, kind, actor, payload_json
+```
+
+- `seq` is monotonic per task.
+- `actor ∈ {agent, user, chatbot, system, webhook}`.
+- `kind ∈ {state_change, agent_message, clarification_request, clarification_answer, human_message, summary, commit_made, subtask_spawned, cost_report, execution_started, execution_ended}`.
+- Granularity: **milestones, not raw stream chunks.** Raw Claude stream-json stays in `stdout.log` on disk, linked from the `execution_started` event.
+- Replay: read events in `seq` order, fold state transitions, reconstruct UI. **For humans, not for re-prompting the agent.** Cross-model agent replay is explicitly out of scope.
+
+### 3. Per-runner structured agent channel (no more file conventions)
+
+Today the agent uses `CLAUDOMATOR_QUESTION_FILE`, `CLAUDOMATOR_SUMMARY_FILE`, and `POST /api/tasks` for subtasks — orchestrated by a ~1KB prompt preamble. We replace this with a `Runner` interface that exposes normalized callbacks:
+
+```go
+type AgentChannel interface {
+    OnAskUser(question string, schema any) (answer string, err error)
+    OnReportSummary(text string)
+    OnSpawnSubtask(spec TaskSpec) (taskID string, err error)
+    OnRecordProgress(message string)
+}
+```
+
+Each runner picks its own wire:
+- **ClaudeRunner:** registers an in-process MCP server via `--mcp-config`. Tools (`ask_user`, `report_summary`, `spawn_subtask`, `record_progress`) bound to a per-task scoped token minted at subprocess spawn. Server derives `task_id` from token; never trusts agent-supplied IDs.
+- **GeminiRunner:** uses Gemini CLI tool-use registration (or direct API if CLI tool-use proves unworkable — to determine during build).
+- **LocalRunner:** uses Ollama-style tool-use API. Falls back to file conventions only if a future runner lacks tool-use plumbing.
+
+**Safety tradeoffs accepted:**
+- Atomicity at run boundary: MCP calls land mid-run, file-based intent lands at exit. Mitigated by designing tools idempotent (`report_summary` upserts, `ask_user` creates a clarification event which is fine mid-run by design).
+- Blast radius: per-task token scopes the agent's access to one task's data.
+- Portability: per-runner transport means non-MCP runners stay supported via callback abstraction.
+
+### 4. Web UI = observability
+
+**In:** task tree view, live log tailing, accept/reject buttons, event-stream timeline per task, budget headroom display.
+
+**Out:** manual task creation form, elaborate textbox, stories dashboard.
+
+Chatbots create tasks via the MCP server. Humans visit the web UI to watch, intervene on `READY`, or answer a `BLOCKED` clarification when push notification pings them.
+
+### 5. Three runners + Classifier
+
+| Runner | Status | Transport | Resume mechanism |
+|---|---|---|---|
+| ClaudeRunner | exists, works | `claude -p` subprocess + MCP | provider-side via `--resume <session_id>` |
+| GeminiRunner | scratch rewrite (current stub deleted) | `gemini` CLI subprocess + tool-use registration | message-history re-feed |
+| LocalRunner | exists (Phase 1 of `local-oss-runner.md`), extend with agent-channel | OpenAI-compat HTTP + tool-use | message-history re-feed |
+
+Classifier stays. Its job becomes: given a task, choose Claude-cloud vs Gemini-cloud vs Local, and within Claude/Gemini, the specific model. **Budget state feeds this decision** (#7).
+
+### 6. Auth: loopback default + reverse proxy + scoped tokens
+
+- Server binds to `127.0.0.1` by default. Refuses to bind non-loopback unless `external_bind_allowed = true` is set in config — fail loud on misconfiguration.
+- External network access: user fronts with Tailscale / Cloudflare Access / Caddy. Out of our code, battle-tested.
+- Internal tokens, each scoped:
+  - **GitHub webhooks:** HMAC-SHA256 (already exists).
+  - **Chatbot MCP + web UI:** one shared bearer token from config; user pastes into MCP client config. One user, one credential.
+  - **Agent MCP back-channel:** per-task token minted at subprocess spawn; bound to single task ID; expires at task termination. Server resolves `task_id` from token — never accepts agent-supplied `task_id`.
+  - **Web push:** standard VAPID subscriptions.
+
+### 7. Budget — multi-dimensional
+
+- Per-task `max_budget_usd` (exists).
+- **Rolling 5-hour budget per provider** (Claude, Gemini) — track spend in a window. Dispatcher refuses to start a paid task that would breach the window. Classifier prefers local when headroom is thin.
+- Optional per-day / per-week hard ceiling in config.
+- Local runners bypass entirely — free.
+- UI surfaces headroom: e.g. "Claude: 67% remaining in current 5h window."
+
+This is genuinely new code: budget accountant, classifier hook, UI surface. Not throwaway.
+
+### 8. Resume = per-runner sovereign opaque state
+
+Each runner persists its own resume state as opaque JSON in `executions.runner_state_json`. The Runner interface:
+
+```go
+Resume(ctx context.Context, opaqueState []byte) (*Result, error)
+```
+
+- **ClaudeRunner** stores `{session_id}`, calls `--resume`. **Prompt caching survives** — this is the killer feature that justifies the special case. Cost can differ 10x on long resumed tasks.
+- **GeminiRunner** stores `{messages: [...]}`, re-feeds on resume.
+- **LocalRunner** stores `{messages: [...]}`, re-feeds on resume.
+
+The "specialness" of Claude is entirely internal to ClaudeRunner — the dispatcher and event store don't know. Cross-model resume is **not** a goal; if we ever want it, revisit then.
+
+### 9. Subscription model: push, agent-return-friendly
+
+Server broadcasts events on WebSocket (live UI) + web push (high-signal subset: `clarification_request`, terminal state changes, `summary`). Subscription record:
+
+```
+subscriptions: id, endpoint, kind_filter, task_filter, actor_label
+```
+
+Today's only subscribers are browser WebSocket clients and web-push endpoints. The model is flexible enough that adding an autonomous-chatbot subscriber later is one row: `{endpoint: chatbot_url, kinds: ["clarification_request"], actor_label: "auto-responder"}`. The `POST /api/tasks/{id}/answer` endpoint already accepts any caller; the `actor` field on the resulting `clarification_answer` event records who answered.
+
+---
+
+## What Gets Deleted
+
+| Surface | Reason |
+|---|---|
+| `stories` table + handlers + UI dashboard | Folded into events + task tree view |
+| `tasks.question_json`, `interactions_json`, `summary`, `elaboration_input` | Replaced by events table |
+| `POST /api/tasks/elaborate` endpoint | Chatbots elaborate in-conversation; no REST shell-out path |
+| `sanitizeElaboratedTask` keyword heuristics | Same |
+| Web UI: create-task form, elaborate textbox, stories dashboard | Out of scope for observability UI |
+| `CLAUDOMATOR_QUESTION_FILE`, `CLAUDOMATOR_SUMMARY_FILE` env conventions | Replaced by per-runner agent channel |
+| GeminiRunner stub (`internal/executor/gemini.go` simulated output) | Scratch rewrite as real runner |
+| Planning preamble's file-protocol section | Shrunk to "use your tools" — runners register tools per-transport |
+
+## What Gets Built
+
+| Surface | Notes |
+|---|---|
+| `internal/event` package | `Event` struct, kind constants, `EventStore` interface |
+| `events` table + scan/CRUD methods | Additive migration; indexed on `(task_id, seq)` |
+| Per-task scoped MCP server | Hosted inside claudomator process; tokens minted per subprocess spawn |
+| Chatbot-facing MCP server | Tools: `submit_task`, `list_tasks`, `get_task`, `get_events`, `answer_question`, `accept_task`, `reject_task`, `cancel_task` |
+| `AgentChannel` interface + per-runner implementations | Normalized callbacks; transport hidden in runner |
+| Real GeminiRunner | Subprocess + stream parser + sandbox integration; mirrors ClaudeRunner shape |
+| Budget accountant | Rolling 5h window per provider; classifier hook; UI surface |
+| Loopback-default binding + `external_bind_allowed` flag | Hard refusal on misconfig |
+| Per-task token minting + scoping | Auth context for agent MCP calls |
+| Web UI event-stream timeline component | Replaces ad-hoc question/summary panels |
+| `external_refs` field on tasks | For opportunistic tracker back-sync |
+
+---
+
+## Phasing
+
+These are sequencing notes, not commitments to PR boundaries. Each phase should be independently shippable.
+
+**Phase 1 — Events table + Runner interface refinement.** Add `events` table, `EventStore`, write paths for state changes and existing question/summary flows. Refactor `Runner` to expose `AgentChannel` callbacks; ClaudeRunner keeps file-based wire as transitional default. No user-visible behavior change yet.
+
+**Phase 2 — ClaudeRunner MCP agent channel.** Per-task MCP server, token minting, four agent tools. Delete file-based escape hatches. Shrink preamble. Migration: existing in-flight tasks finish on the old wire; new tasks use MCP.
+
+**Phase 3 — Chatbot MCP server + UI rebuild.** Chatbot-facing MCP server. Web UI gets event-stream timeline, drops create form / elaborate / stories. Elaborate REST endpoint removed.
+
+**Phase 4 — GeminiRunner scratch rewrite.** Real subprocess + stream parser + sandbox + tool-use registration. Plug into AgentChannel.
+
+**Phase 5 — LocalRunner agent channel.** Extend the existing LocalRunner (from `local-oss-runner.md`) to participate in AgentChannel via Ollama tool-use. Was previously fire-and-forget; now first-class for orchestration tasks.
+
+**Phase 6 — Budget accountant.** Rolling 5h window tracking, classifier integration, UI headroom display, dispatcher gating.
+
+**Phase 7 — Auth tightening.** Loopback default, `external_bind_allowed` flag, refuse-bind enforcement, chatbot bearer rotation flow.
+
+**Phase 8 — Cleanup pass.** Delete `stories`, deprecated columns, dead handlers, dead UI. Update CLAUDE.md and ADRs.
+
+---
+
+## Open Design Questions (resolve at phase entry)
+
+- **MCP tool schemas.** Exact arg shapes for `ask_user` (free text? JSON schema for typed answers?), `spawn_subtask` (how much of `TaskSpec` is exposed?), `submit_task` (how to handle `project_dir` resolution from chatbot side?). Defer to Phase 1/3 detailed design.
+- **Gemini transport for tool-use.** Whether `gemini` CLI's tool-use is reliable enough or we need to drop to the SDK. Decide in Phase 4 spike.
+- **Web push subset.** Final list of event kinds that trigger push vs WebSocket-only. Default: `clarification_request`, `state_change → READY/FAILED/BLOCKED`, `summary`. Tune in Phase 3.
+- **Single shared bearer token vs per-client.** If multi-device gets painful, revisit. Defer.
+- **Budget hard-ceiling defaults.** What's the user's actual daily Claude spend tolerance? Surface in config with no default; require explicit value.
+
+## Deferred (revisit only if pressure appears)
+
+- Cross-model resume.
+- Non-code-task UX (event types for "agent produced text", task-with-conversation-as-primary view). Shape must remain possible; no code or docs accommodate it.
+- Autonomous chatbot answering clarifications. Subscription model supports it; no implementation.
+- GitLab webhook handler (parallel to GitHub). Add when needed.
+- Todoist or similar personal-task-tracker bridges. Same shape as GitHub webhook; add when needed.