claudomator.git/internal, branch fix/dockersandbox-gitpush-host-security

fix(sandbox): run GitPush from host to fix local-remote push + harden against sandbox escape

2026-07-04T09:43:11+00:00

DockerSandbox.GitPush was running `git push` inside the container via `docker exec`, but the origin remote URL is a host filesystem path that was never bind-mounted into the container — only hostDir is. This caused `exit status 128` in TestDockerSandbox_RealContainer_Lifecycle. Fix: run `git -C hostDir push origin -- ` from the host (mirroring HostSandbox.GitPush), so local-path remotes are reachable. Two security flags prevent a compromised workspace from escaping onto the host: - `-c core.hooksPath=/dev/null` neutralises any hooks planted in .git/ - `-c protocol.ext.allow=never` blocks the ext:: pseudo-protocol Also rejects refs starting with "-" and inserts "--" before the ref to prevent argument injection. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_01DNDHfCtTZDbQueEV5sUBiD

feat(web,api): add Budget/Roles dashboard -- final phase of the harness redesign (Phase 9b)

2026-07-04T08:50:46+00:00

Two new tabs, matching Phase 9a's conventions: Budget (per-provider spend meters, escalation funnel, spend-over-time) and Roles (version history, draft activation, readable escalation-ladder view) -- the human-facing surface for the token-husbanding value proposition and the Phase 5/8 versioned role-config system. New backend (minimal, additive, matching existing endpoint conventions -- unauthenticated like /api/budget and /api/roles/*): - internal/storage/dashboard.go: QueryEscalationFunnel (executions grouped by escalation_rung + agent, count + cost) and QuerySpendTimeseries (cost per provider bucketed hourly/daily, mirroring QueryDashboardStats' existing bucketing expressions). Documented honestly: rung 0 is not exclusively "resolved locally" -- escalation_rung defaults to 0 uniformly, so non-role-typed executions (which never climb a ladder) land there too, alongside role-typed tasks genuinely resolved at tier 0. A role-only variant would need a join against tasks.config_json with no queryable role column on executions -- intentionally out of scope. - internal/api/dashboard.go: GET /api/escalation-funnel, GET /api/spend-timeseries (both ?window=5h|24h|7d|, default 24h). - internal/storage.ListRoleNames + GET /api/roles: the "which roles exist" gap -- there was no way to discover role names before this, only to list versions for a role you already knew the name of. Chart-form decisions (dataviz skill, invoked before writing chart code): horizontal stacked bar for the escalation funnel (rung order already encodes the funnel shape positionally; color only needed for per-provider segments within each rung); multi-line for spend-over-time; a fixed-order categorical palette from the skill's validated palette.md slots for provider identity (re-validated against this app's dark surface, passing); a separate status (good/warning/critical) palette for budget meters, deliberately distinct from both --state-* and the provider palette. Caught and fixed a real bug during visual QA: converging near-zero end-labels on the spend chart were overlapping (an anti-pattern the skill explicitly flags) -- fixed with a 14px minimum-gap check before direct-labeling an endpoint, leaning on the legend/tooltip otherwise. Verified with a real running server, real seeded data (65 executions across rungs 0-2 with a realistic provider mix, 2 roles with active/draft/retired role_configs versions written directly via internal/storage), and a real headless-browser session (reusing Phase 9a's Chromium/proxy scaffinding): confirmed correct rung totals/percentages, provider legends, a 3-line spend chart, live window-selector re-render, correct active-version highlighting on the role panel, and a real Activate click on a draft version -- verified via both DOM re-render and a direct backend GET that it truly persisted. go build/vet/test -race -count=1 all pass, full suite. node --test web/test/*.mjs: 291/291 passing (16 new). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs

feat(story,role): add retro ceremony -- closes the self-improvement loop (Phase 8)

2026-07-04T05:05:36+00:00

The final mechanism the versioned role-config model (Phase 5) was built for: when a story reaches DONE, StoryOrchestrator spawns a retro-role task that reflects on the story's full history and proposes draft role_configs versions for a human to review and activate via the existing (unchanged) POST /api/roles/{role}/activate. - AgentChannel gains a 6th method, ProposeRoleConfig(ctx, role.RoleConfig) (version, err), following ProposeEpic's precedent (Phase 7c): a structured tool call, not summary-parsing. storeChannel.ProposeRoleConfig calls the same Store.CreateRoleConfig the human-facing POST /api/roles/{role}/versions endpoint already uses (proposed_by: "retro"), landing a new draft row without touching whatever's currently active. Wired through both transports exactly like ProposeEpic: internal/agentloop/tools.go (native loop) and internal/executor/agentmcp.go (MCP). - StoryOrchestrator.Tick now routes a story at status DONE to a new processRetro stage instead of processStory -- a sibling stage, not a continuation, since the Builder->Evaluators->Arbitration chain is long settled by then. processRetro only *reads* that settled pipeline (read-only findEvaluators/findArbitration counterparts to ensureEvaluators/ensureArbitration -- it never spawns/mutates Builder-pipeline tasks) to locate the Arbitration task the retro task depends on, then spawns (idempotently -- checks for an existing retro-role dependent first) one retro-role task with instructions assembled from the story's spec/acceptance-criteria, full task tree, per- task cost/escalation history, active role_configs per role encountered, and the story's own event stream (evaluator verdicts, arbitration decision). - event.KindRetroCaptured (attached to the story's ID, matching KindEvalVerdict/KindArbitrationDecided's convention) fires once the retro task completes (auto-accepted like every other pipeline task), aggregating every event.KindRoleConfigProposed the retro task recorded (one per propose_role_config call) into {task_id, proposals: [{role, version}], summary} -- the summary is the "capturing lessons" half of this ceremony, the proposals are the versioned-config half. - Human activation is completely untouched: drafts land through the identical CreateRoleConfig/config_json path Phase 5's endpoints already handle, confirmed via existing role-endpoint tests passing unmodified. go build/vet/test -race -count=1 all pass, full suite (20 packages) -- one run hit a known, pre-existing, intermittent flake under full-suite load (unrelated to this phase's files) that did not reproduce on two immediate reruns, both in isolation and full-suite. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs

feat(story,scheduler): add epic-proposal tool + AskUser-timeout escalation (Phase 7c)

2026-07-04T04:39:28+00:00

Two independent pieces, completing Phase 7. Epic-proposal tool: AgentChannel gains a 5th method, ProposeEpic(ctx, EpicProposal{Name, Description, StoryIDs}) (epicID, err), implemented on storeChannel -- matches an existing epic by exact name or creates one (DiscoverySource: "agent"), sets epic_id on each resolvable story (skips, doesn't fail, on an unresolved ID), emits KindEpicProposed attached to the epic's own ID with payload {epic_id, name, story_ids}. Wired into both transports exactly like Phase 6 wired role into spawn_subtask: a new propose_epic tool in the native tool-use loop (internal/agentloop/tools.go) and the MCP transport (internal/executor/agentmcp.go). This is the mechanism for a discovery/planner-role agent to act on its own judgment that several stories it's been given form one cohesive initiative -- the judgment itself lives in the calling agent's instructions/model, not in this code. AskUser-timeout escalation: extends the existing Scheduler (Phase 5's retry-then-escalate watcher) rather than adding a new component, since "stuck task needs escalation" is exactly what it already does. Finds role-typed BLOCKED tasks whose question has been outstanding longer than SchedulerConfig.AskUserTimeoutSeconds (default 10 minutes) using task.UpdatedAt as the outstanding-since timestamp -- no new column needed, since UpdateTaskQuestion already stamps it the instant a question is recorded and nothing else touches the row while BLOCKED. Resolves the next ladder tier from the latest execution's EscalationRung, records the system-authored fallback answer as an audit-trail task.Interaction, clears the question, sets the new tasks.needs_review flag, emits KindEscalated (now carrying a trigger field: "failure" vs "ask_user_timeout" for the existing failure-retry path vs this one), and resumes via Pool.SubmitResume at the escalated tier -- degrading to same-tier resume with final:true if the ladder's exhausted or no role config exists, since unblocking the task takes priority over having somewhere higher to escalate to. GET /api/tasks?needs_review=true surfaces auto-decided tasks for human review. go build/vet/test -race -count=1 all pass, full suite (20 packages), run twice to rule out flakiness in the new tests. (One pre-existing, unrelated test -- TestHandleRunTask_CascadesRetryToFailedDeps, a tempdir-cleanup race -- appeared once under full-suite load per the implementing agent's report and did not reproduce in this verification's runs either; not a regression from this work.) Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs

feat(story): add StoryOrchestrator -- Builder->Evaluators->Arbitration->accept (Phase 7b)

2026-07-04T04:08:41+00:00

A deterministic, poll-based watcher (internal/scheduler.StoryOrchestrator, sibling to the Phase 5 Scheduler) that drives a story.Story through its execution pipeline, rather than relying on an LLM agent to correctly orchestrate its own fan-out via tool calls. Mechanism: polling, not a handleRunResult hook. Every task the orchestrator watches (a story's root/Builder task, 4 Evaluators, Arbitration) is top-level (no ParentTaskID), and executor.Pool.handleRunResult only ever lands a top-level task at READY or BLOCKED -- never COMPLETED directly, since that transition normally requires a human/chatbot POST /api/tasks/{id}/accept in a different package. A handleRunResult hook would never observe it; polling doesn't care how/whether a task reached a given state. Stages: Builder COMPLETED -> spawn 4 role-typed Evaluator tasks (evaluator_quality/security/correctness/performance, DependsOn: [builder], no ParentTaskID -- true DAG siblings, not delegated subtasks) + story -> VALIDATING. Each Evaluator COMPLETED -> emit KindEvalVerdict (attached to the story's ID, so one GET /api/stories/{id}/events call surfaces every verdict). All 4 Evaluators COMPLETED -> spawn 1 Arbitration task (role: planner, DependsOn: all 4 evaluator IDs). Arbitration COMPLETED -> emit KindArbitrationDecided, story -> REVIEW_READY. POST /api/stories/{id}/accept (mirrors handleAcceptTask) -> DONE, emits KindHumanAccepted. Fixes a gap caught before merging: since none of Builder/Evaluators/ Arbitration have a ParentTaskID, none of them auto-complete -- each would otherwise need a separate manual /api/tasks/{id}/accept, meaning 6 human clicks per story before ever reaching the intended single story-level gate. StoryOrchestrator.autoAccept now transitions each of these specific tasks READY->COMPLETED itself (via the same validated Store.UpdateTaskState path acceptTask uses), scoped only to tasks already established as part of a story's pipeline (root task, or role-matched dependents from ensureEvaluators/ensureArbitration) -- never a blanket sweep of unrelated READY tasks. This makes POST /api/stories/{id}/accept the system's only required human touchpoint for the whole chain, matching the design goal that story (not task/subtask) is the human-interaction atom. Idempotency: structural for task-creation stages (ensureEvaluators/ ensureArbitration check ListDependents for already-existing role-matched tasks before creating -- crash/restart-safe); story.Status=="VALIDATING" gates the Arbitration->REVIEW_READY write (nothing further downstream to check structurally there); an in-memory handledVerdicts set (mirrors Scheduler.handled) dedupes per-evaluator KindEvalVerdict emission across poll ticks, resetting harmlessly on restart. Documented simplification: finalizeArbitration never parses the Arbitration summary for approve/reject -- always routes to REVIEW_READY; NEEDS_FIX is manually settable via PUT /api/stories/{id}. A later phase could close this with a dedicated verdict-reporting AgentChannel method instead of parsing free text. go build/vet/test -race -count=1 all pass, full suite (20 packages). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs

feat(story): add Epic/Story data model + REST CRUD (Phase 7a)

2026-07-03T23:46:26+00:00

Pure data model + CRUD -- no orchestration behavior yet (that's Phase 7b). Revives ADR-007's stories concept (adapted; ADR-007 itself was deleted as "more machinery than the usage pattern needed") while staying lean: reuses the existing events/projects/task-tree machinery rather than building a parallel hierarchy. - internal/story: Epic/Story types. Epic is a loosely-scoped initiative (OPEN/CLOSED, no execution semantics). Story is a shippable slice of work realized by a task tree rooted at root_task_id; epic_id/project_id/ root_task_id are loose references (no FK enforcement, matching the codebase's existing tolerance for unmatched reference strings like tasks.project). - storage: new epics/stories tables (additive migrations) + CRUD methods. Story status is intentionally unvalidated pass-through in this phase -- no task.ValidTransition-style state machine, since there's no orchestrator yet to make transitions meaningful. - event: 9 new Kind constants for the ceremony this layer will eventually drive (epic_proposed, discovery_proposed, framing_decided, groomed, prioritized, eval_verdict, arbitration_decided, retro_captured, human_accepted) -- defined but nothing emits them yet. - api: GET/POST /api/epics, GET/PUT /api/epics/{id}, GET /api/epics/{id}/stories, GET/POST /api/stories, GET/PUT /api/stories/{id}, and GET /api/stories/{id}/task-tree -- BFS walk from root_task_id following both parent_task_id children and depends_on-edge dependents (visited-set guarded), returning a flat node list for a later UI phase to render as a graph. Unauthenticated, matching the existing projects/tasks endpoints' posture. go build/vet/test -race -count=1 all pass, full suite. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs

feat(executor): add DAG auto-cascade-fail + role-typed subtask spawning (Phase 6)

2026-07-03T23:22:42+00:00

Two prerequisites for safe parallel evaluator fan-out (later phase): 1. Auto-cascade-fail: previously, a failed task's dependents just sat PENDING/QUEUED forever (or until something eventually tried to dispatch them and discovered the dependency was dead). Pool.cascadeFail now fires right after a task lands in a terminal failure state (FAILED/TIMED_OUT/ CANCELLED/BUDGET_EXCEEDED, from handleRunResult, the budget-gate reject path, and the checkDepsReady dependency-failure path), recursively cancelling every not-yet-run dependent (PENDING/QUEUED only -- RUNNING and terminal states are left alone) with a message referencing the upstream failure. A visited-set guards recursion, which turned out to be load-bearing rather than defense-in-depth: task creation does not prevent dependency cycles anywhere in this codebase. Correction to an earlier assumption: internal/executor's waitForDependencies is dead code, never called. The live mechanism is checkDepsReady, invoked synchronously in execute() with a self-requeue via time.AfterFunc. Added a freshness re-check (GetTask, bail if no longer QUEUED) at the top of that block so a task cascade-cancelled while sitting in the requeue loop stops silently instead of hitting an invalid CANCELLED->CANCELLED transition or, worse, still getting dispatched. 2. storeChannel.SpawnSubtask hardcoded Agent.Type: "claude" on every spawned child regardless of what role it should play -- a hard blocker for a Planner/Builder task spawning role-typed evaluator subtasks. SubtaskSpec (internal/agentchannel) gains a Role field; when set, the child task gets Agent.Role instead of a hardcoded Type, so Phase 5's role-resolution picks provider/model from that role's escalation ladder. spec.Role == "" (every existing caller) preserves today's exact behavior byte-for-byte -- proven by an explicit regression test, not just new-feature coverage. Threaded the new `role` parameter through both spawn_subtask transports: the native tool-use loop (internal/agentloop/tools.go) and the MCP tool exposed to ContainerRunner-driven claude/gemini agents (internal/executor/agentmcp.go). go build/vet/test -race -count=1 all pass, full suite (20 packages). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs

feat(role): add versioned role configs + escalation ladder + scheduler (Phase 5)

2026-07-03T23:01:50+00:00

Two parts: Part A (fixes a gap from Phase 1): Groq/OpenRouter/OpenAI were documented in docs/api-keys-setup.md as usable once configured, but nothing actually constructed runners for them. internal/cli/cloudrunners.go consolidates anthropic/google/groq/openrouter/openai NativeRunner construction into one table-driven registerCloudRunners() helper, replacing the two hand-written per-provider blocks in serve.go/run.go. Groq/OpenRouter/OpenAI reuse openaicompat (no new adapter code) at SandboxKind: "docker". Part B: the token-husbanding harness's core routing mechanism. - internal/role: RoleConfig/Tier/Rung -- a role's system prompt and a multi-tier (provider, model) escalation ladder, versioned via config_json. - storage: new role_configs table (draft/active/retired, UNIQUE(role, version)) with transactional activate-retires-prior-active semantics; new executions.escalation_rung column. - task.AgentConfig.Role string -- purely additive; every existing task shape (Agent.Role == "") is unaffected, proven by TestPool_Execute_NonRoleTask_Unaffected plus the full pre-existing suite passing unchanged. - executor.Pool.execute(): role-typed tasks with no Agent.Type yet resolve tier 0 of their active ladder (round-robin across multi-candidate tiers, skipping rate-limited providers, falling back to soonest-clearing) before the existing pickAgent/Classifier path runs; SystemPrompt applies to Agent.SystemPromptAppend. Already-resolved role tasks (scheduler resubmits) get their escalation_rung re-derived read-only via findTierIndex. - internal/scheduler: polls role-typed FAILED tasks, retries at the same rung under MaxRetries or escalates to the next tier's first candidate when budget.Accountant.Allow() permits (emitting event.KindEscalated), else leaves the task FAILED with a final:true KindEscalated event. An in-memory per-execution-ID "handled" set keeps the poll loop convergent. Started by `serve` only, config knob [scheduler].poll_interval_seconds. - internal/api: POST/GET /api/roles/{role}/versions, POST /api/roles/{role}/activate -- unauthenticated, matching the existing projects/tasks REST endpoints' auth posture (only chatbot MCP, agent MCP, and WebSocket are api_token-gated in this codebase today). Documented as stored-but-not-yet-enforced (CLAUDE.md Design Debt, matching how task.Priority/RetryConfig are already documented): RoleConfig.Tools/ SandboxKind don't affect dispatch yet; DefaultBudgetUSD is read narrowly as the scheduler's escalation cost estimate, not enforced at initial dispatch; scheduler escalation always targets Candidates[0] (no round-robin, unlike initial-dispatch tier-0 resolution); the scheduler's dedupe is per-process and resets on restart (idempotent, harmless). go build/vet/test -race -count=1 all pass, 21 packages. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs

feat(provider): add native Google Gemini API adapter (Phase 4)

2026-07-03T22:33:22+00:00

Adds internal/provider/google, the second native cloud adapter (following internal/provider/anthropic's pattern) on top of Phase 1's provider-neutral tool-use loop, wired to a Docker-sandboxed NativeRunner under agent.type: "google" -- a separate execution path and budget bucket from the existing CLI-subprocess "gemini" ContainerRunner, which is untouched. Wire-format research (the highest-risk part of this adapter): Gemini's multi-turn function-calling shape was resolved by cross-referencing the REST API reference's own generateContent example against the go-genai SDK's struct tags on GitHub -- both agree on functionCall/functionResponse parts keyed by "name" (with an optional "id" for round-tripping ToolCall.ID), with the response fed back inside a "user"-role Content (Gemini has no tool/function role, mirroring Anthropic's lack of one). A separate fetched source (the function-calling guide page) was deliberately discarded as a reference for this shape -- it documents a different, newer "Interactions API" whose call_id/type:"function_result" structure doesn't fit the contents/parts/candidates shape used everywhere else. - internal/provider/google: request/response translation, systemInstruction handling, role mapping (assistant->model, tool-results->user role), per-model-prefix pricing table (2.5 Pro/Flash/Flash-Lite, 2.0, 1.5 tiers) - internal/retry: IsRateLimitError additively extended for RESOURCE_EXHAUSTED - internal/config: RunnersConfig.Google/GoogleEnabled() - internal/cli/serve.go, run.go: runners["google"] construction mirroring the Anthropic wiring exactly (Docker sandbox default) - docs/api-keys-setup.md: Google marked wired-up, budget-bucket/disable/ verify guidance added matching the Anthropic section go build/vet/test -race all pass. No live Gemini API key available in this environment; verified via fake-httptest-server adapter tests (plain text, tool-use round-trip, multi-turn tool-result, rate-limit error matching) plus a Pool/NativeRunner routing test. Live E2E is a follow-up once a key is configured, same as Phase 2's Anthropic adapter. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs

feat(sandbox): add DockerSandbox + pre-tool-use guardrail hooks (Phase 3)

2026-07-03T09:21:32+00:00

Gives native-API-driven agents (currently just the Phase 2 Anthropic adapter) real container isolation, decoupled from model invocation -- the model call happens in the Go process via provider.Provider, only tool execution happens in the container, unlike the CLI-subprocess ContainerRunner (left completely untouched) where the claude/gemini CLI runs inside the container. - internal/sandbox/dockersandbox.go: Sandbox via a long-lived `docker run -d ... sleep infinity` container (started once per execution, not per tool call), host-side git clone + bind-mount matching ContainerRunner's existing pattern, docker exec for read/write/bash/glob. Reuses images/agent-base (claudomator-agent:latest) rather than standing up a second image. WorkDir()/resume persists the host bind-mount directory (matching HostSandbox's contract); a resumed sandbox lazily starts a fresh container against that directory rather than trying to reattach to a possibly-gone one. - internal/sandbox/guard.go, hooks.go: Hook interface (CheckBash/CheckWrite), Guarded wrapper, DenylistBashHook (rm -rf /, force-push, curl|sh, sudo, chmod 777, dd if=) and ProtectedPathHook (.git/**, .env*, credentials/, .github/workflows/**). A rejection returns *RejectionError, which agentloop/tools.go now recognizes and feeds back to the model as a normal (non-fatal) tool-error result instead of aborting the run. - NativeRunner wraps whichever Sandbox it builds (Host or Docker) in Guarded{Hooks: DefaultHooks()} uniformly. The "anthropic" runner now uses DockerSandbox; "local" stays on HostSandbox by design (local models are the harness's more-trusted, lower-stakes-to-run tier). Docker is not installed in this dev environment (no docker/podman/containerd on PATH), so DockerSandbox's real container-lifecycle behavior is verified via mocked-command unit tests only -- go test -race ./... passes throughout, with the two real-daemon integration tests gated behind a dockerAvailable(t) check and skipping here. Live verification against an actual Docker host is a follow-up before relying on the "anthropic" agent type in production. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V1moSNCJRcP6kykA4tyUSs