path: root/docs/workflow-and-entry-points.md

# Claudomator: Workflow & Entry Points

This document diagrams Claudomator's overall workflow, all entry points into the
task-execution pipeline, the task state machine, and where the recursive
arbitrated-review mechanism sits relative to that state machine.

---

## 1. Entry Points Overview

Every path that ultimately executes work passes through `executor.Pool`, the single
choke-point that bounds concurrency and gates spend. Some entry points submit
directly; others only write storage and let a poll loop or a human /run call submit.

```mermaid
flowchart TD
    subgraph CLI["CLI (claudomator run)"]
        CLI_RUN["claudomator run <file.yaml>\n(internal/cli/run.go)\nParses YAML, creates + submits directly"]
    end

    subgraph REST["REST API"]
        API_CREATE["POST /api/tasks\n(creates PENDING task in storage only)"]
        API_RUN["POST /api/tasks/{id}/run\n(PENDING → QUEUED + Pool.Submit)"]
        API_CREATE --> API_RUN
    end

    subgraph CHATBOT["Chatbot MCP (/chatbot/mcp)"]
        CB_SUBMIT["submit_task tool\n(creates + submits in one step)"]
        CB_STORY["create_story tool\n(creates Story row in storage only;\nStoryOrchestrator spawns Builder on next tick)"]
        CB_ACCEPT["accept_story tool\n(REVIEW_READY → DONE;\norchestrator triggers retro next tick)"]
    end

    subgraph WEBHOOK["GitHub Webhook"]
        WH["POST /api/webhooks/github\ncheck_run / workflow_run failure\n(internal/api/webhook.go)\ncreates PENDING task in storage;\ndoes NOT auto-submit to pool"]
    end

    subgraph STORY_API["Story REST API"]
        ST_CREATE["POST /api/stories\n(creates Story row in storage only)"]
        ST_ACCEPT["POST /api/stories/{id}/accept\n(REVIEW_READY → DONE)"]
    end

    subgraph AGENT_TOOLS["Agent Back-channel Tools\n(MCP /mcp or native tool-use loop)"]
        AG_SPAWN["spawn_subtask\n(creates PENDING child task;\nparent_task_id always set;\ndoes NOT auto-submit to pool)"]
        AG_EPIC["propose_epic\n(creates/updates Epic row;\ngroups Stories; storage only)"]
        AG_RC["propose_role_config\n(creates draft role_configs row;\nstorage only)"]
    end

    subgraph ORCHESTRATOR["StoryOrchestrator (poll, 15 s)\n(internal/scheduler/story_orchestrator.go)"]
        SO_EVAL["ensureEvaluators\n(4 Evaluator tasks per builder node)"]
        SO_ARB["ensureArbitration\n(1 Arbitration/planner task)"]
        SO_FIX["ensureFixAttempt / spawnNestedFixAttempt\n(new builder task after rejection)"]
        SO_RETRO["processRetro\n(retro task after story DONE)"]
    end

    subgraph SCHEDULER["Scheduler (poll, ~10 s)\n(internal/scheduler/scheduler.go)"]
        SC_FAIL["processTask\n(retry / escalate FAILED role-typed tasks)"]
        SC_TIMEOUT["tickAskUserTimeouts\n(auto-answer + resume BLOCKED tasks\noutstanding > ask_user_timeout)"]
    end

    STORAGE[("SQLite storage\n(tasks table)")]
    POOL(["executor.Pool\n(bounded dispatcher)"])

    %% CLI path — direct submit
    CLI_RUN -->|"creates + QUEUED + Pool.Submit()"| POOL

    %% REST path — two-step
    API_RUN -->|"Pool.Submit()"| POOL

    %% Chatbot MCP — submit_task is direct; story tools are storage-only
    CB_SUBMIT -->|"creates + QUEUED + Pool.Submit()"| POOL
    CB_STORY -->|"Story row only"| STORAGE
    CB_ACCEPT -->|"Story REVIEW_READY → DONE"| STORAGE

    %% Webhook — creates PENDING task only; needs /run to execute
    WH -->|"creates PENDING task"| STORAGE

    %% Story REST — storage-only; orchestrator picks up on next tick
    ST_CREATE -->|"Story row only"| STORAGE
    ST_ACCEPT -->|"Story REVIEW_READY → DONE"| STORAGE

    %% Agent tools — spawn_subtask is storage-only; propose_epic/role_config are pure storage
    AG_SPAWN -->|"creates PENDING child task\n(scripts/start-next-task,\nStoryOrchestrator, or /run\nsubmit it later)"| STORAGE
    AG_EPIC -->|"Epic/Story rows only"| STORAGE
    AG_RC -->|"role_configs draft row only"| STORAGE

    %% StoryOrchestrator spawns tasks AND submits them
    STORAGE -->|"StoryOrchestrator polls stories;\nspawnRoleTask: CreateTask + QUEUED + Pool.Submit()"| SO_EVAL
    SO_EVAL -->|"Store.CreateTask + Pool.Submit()"| POOL
    SO_ARB -->|"Store.CreateTask + Pool.Submit()"| POOL
    SO_FIX -->|"Store.CreateTask + Pool.Submit()"| POOL
    SO_RETRO -->|"Store.CreateTask + Pool.Submit()"| POOL

    %% Scheduler → Pool
    SC_FAIL -->|"Pool.Submit() or Pool.SubmitResume()"| POOL
    SC_TIMEOUT -->|"Pool.SubmitResume()"| POOL

    %% Pool outcomes feed the schedulers
    POOL -->|"task reaches READY/COMPLETED/FAILED;\norchestrator polls storage"| ORCHESTRATOR
    POOL -->|"task reaches FAILED;\nscheduler polls"| SCHEDULER
    POOL -->|"task stays BLOCKED;\nscheduler polls (ask_user timeout)"| SCHEDULER
```

> **Note on PENDING subtasks:** When an agent calls `spawn_subtask`, the child task
> lands in PENDING state in storage. Nothing auto-submits it. Dispatch happens via one
> of: (a) the `scripts/start-next-task` helper (calls `POST /api/tasks/{id}/run`),
> (b) `StoryOrchestrator.spawnRoleTask` if the subtask is a story-pipeline role task,
> (c) a human or chatbot calling `POST /api/tasks/{id}/run`, or
> (d) `RecoverStaleQueued` on server restart for tasks already in QUEUED state.

---

## 2. Detailed Entry Point Reference

| Entry Point | Code Location | What It Creates | Who Submits to Pool |
|---|---|---|---|
| `claudomator run <file>` | `internal/cli/run.go` | Task(s) from YAML | `cli` directly: `CreateTask` → `UpdateTaskState(QUEUED)` → `Pool.Submit` |
| `POST /api/tasks` + `POST /api/tasks/{id}/run` | `internal/api/server.go`, `taskops.go` | PENDING task, then QUEUED | `handleRunTask` via `Pool.Submit` |
| Chatbot MCP `submit_task` | `internal/api/chatbotmcp.go` | Task (PENDING→QUEUED in one step) | `submitTask()` via `Pool.Submit` |
| Chatbot MCP `create_story` | `internal/api/chatbotmcp.go` | Story row in storage | `StoryOrchestrator` (next tick): creates + submits Builder task |
| Chatbot MCP `accept_story` | `internal/api/chatbotmcp.go` | Story status DONE in storage | `StoryOrchestrator` (next tick): spawns retro task |
| `POST /api/stories` | `internal/api/stories.go` | Story row in storage | `StoryOrchestrator` (next tick) |
| `POST /api/stories/{id}/accept` | `internal/api/stories.go` | Story status DONE in storage | `StoryOrchestrator` (next tick): spawns retro task |
| `POST /api/webhooks/github` | `internal/api/webhook.go` | Task in **PENDING** state only | Human/chatbot/script must call `POST /api/tasks/{id}/run` |
| Agent `spawn_subtask` (no `role=`) | `internal/executor/agentmcp.go`, `internal/agentloop/tools.go` | Child task with `parent_task_id`, `agent.type=claude` | **Not auto-submitted**; parent goes BLOCKED; child is dispatched later |
| Agent `spawn_subtask` (with `role=`) | same | Child task with `parent_task_id`, `agent.role` set, no Type/Model | **Not auto-submitted**; role resolved at eventual dispatch via `Pool.execute()` |
| Agent `propose_epic` | `internal/executor/agentmcp.go`, `internal/agentloop/tools.go` | Epic row; links existing stories | Storage only — no task created, no pool call |
| Agent `propose_role_config` | same | Draft `role_configs` version | Storage only — no task created, no pool call |
| `StoryOrchestrator.ensureEvaluators` | `internal/scheduler/story_orchestrator.go` | 4 Evaluator tasks per builder node | `spawnRoleTask`: `CreateTask` → `UpdateTaskState(QUEUED)` → `Pool.Submit` |
| `StoryOrchestrator.ensureArbitration` | same | 1 `planner`-role Arbitration task | same |
| `StoryOrchestrator.ensureFixAttempt` | same | New root builder task after rejection | same |
| `StoryOrchestrator.spawnNestedFixAttempt` | same | New nested builder task after rejection | same |
| `StoryOrchestrator.processRetro` | same | `retro`-role task after story DONE | same |
| `Scheduler.processTask` | `internal/scheduler/scheduler.go` | Re-queues FAILED role-typed task at same or next tier | `Pool.Submit` / `Pool.SubmitResume` |
| `Scheduler.tickAskUserTimeouts` | same | System-authored fallback answer; resumes BLOCKED task | `Pool.SubmitResume` |

---

## 3. Task State Machine

```mermaid
stateDiagram-v2
    [*] --> PENDING : task created\n(POST /api/tasks, webhook,\nspawn_subtask, story rows)

    PENDING --> QUEUED : /run submitted\n(Pool.Submit)

    QUEUED --> RUNNING : dispatcher picks slot

    RUNNING --> READY : execution succeeded\n(exit 0)
    RUNNING --> BLOCKED : agent called ask_user\nor subtasks outstanding
    RUNNING --> FAILED : execution failed\n(non-zero exit / error)
    RUNNING --> TIMED_OUT : timeout exceeded
    RUNNING --> CANCELLED : /cancel called
    RUNNING --> BUDGET_EXCEEDED : spend cap hit

    BLOCKED --> QUEUED : answer provided\n(POST /answer or\nscheduler timeout)
    BLOCKED --> READY : all subtasks COMPLETED\n(maybeUnblockParent)

    READY --> COMPLETED : human/chatbot accept\n(non-builder-role)\nOR\nfinalizeArbitration approves\n(builder-role)
    READY --> PENDING : human/chatbot reject

    FAILED --> QUEUED : /resume or\nScheduler retry/escalate
    TIMED_OUT --> QUEUED : /resume
    CANCELLED --> QUEUED : /resume
    BUDGET_EXCEEDED --> QUEUED : /resume

    COMPLETED --> [*]
```

### Key: READY does not mean "done" for builder-role tasks

For **non**-builder-role tasks (the common case), `READY → COMPLETED` is a
simple human or chatbot accept via `POST /api/tasks/{id}/accept`.

For **builder**-role tasks inside a story pipeline, `READY` means
**"awaiting arbitration"** — the `StoryOrchestrator` intercepts every `READY`
builder node and drives it through the review cycle before ever promoting it to
`COMPLETED`. A builder task at `READY` without a completed arbitration is
deliberately left there until the review is done.

```
builder READY  ──►  ensureEvaluators spawns 4 tasks
                         │
                         ▼
                   4 Evaluators → COMPLETED (auto-accepted)
                         │
                         ▼
                   ensureArbitration spawns 1 planner task
                         │
                         ▼
                   Arbitration → COMPLETED (auto-accepted)
                         │
             ┌───────────┴───────────┐
             │ APPROVED              │ REJECTED
             ▼                       ▼
      builder READY               builder stays READY (superseded)
         → COMPLETED               new fix-attempt task spawned
      (story → REVIEW_READY        (story → NEEDS_FIX → IN_PROGRESS)
       if root node)
```

---

## 4. Recursive Arbitrated-Review Cycle

The review mechanism is **depth-agnostic**: it applies identically at the root
and at any nested depth. `processStory` walks the entire task tree every tick
and applies `processBuilderNode` to every `READY` builder-role node found.

```mermaid
flowchart TD
    subgraph Story["Story (IN_PROGRESS)"]
        ROOT["root_task_id\n(Builder, READY)"]
    end

    subgraph ReviewCycle["Review Cycle (per builder node, any depth)"]
        EVAL["4 Evaluator tasks\nevaluator_quality\nevaluator_security\nevaluator_correctness\nevaluator_performance\n\n(depends_on: builder node)"]
        ARB["1 Arbitration task\nrole: planner\n\n(depends_on: all 4 evaluators)"]
        APPROVE{"Approved?\n(report_verdict event;\ndefault: approve)"}
        FIX["Fix-attempt task\n(same role, new builder;\ndepends_on: rejected node)"]
    end

    subgraph Terminal["Terminal states"]
        COMP["builder node → COMPLETED\n(if root: story → REVIEW_READY)"]
        HUMAN["POST /api/stories/{id}/accept\n(REVIEW_READY → DONE)\n[only manual step in the chain]"]
        RETRO["processRetro: retro-role task\nreflects, proposes role_configs drafts"]
        DONE["story → DONE"]
    end

    ROOT -->|"StoryOrchestrator\nensureEvaluators"| EVAL
    EVAL -->|"all COMPLETED\nensureArbitration"| ARB
    ARB -->|"COMPLETED\nfinalizeArbitration reads\nreport_verdict event"| APPROVE
    APPROVE -->|"Yes"| COMP
    APPROVE -->|"No"| FIX
    FIX -->|"next tick: new node at READY\nrecursive review restarts here"| EVAL
    COMP --> HUMAN
    HUMAN --> DONE
    DONE -->|"processRetro"| RETRO

    NOTE["Nested builder nodes (parent_task_id != ''):\nsame cycle, spawnNestedFixAttempt\ninstead of ensureFixAttempt;\nno story-status change at nested level"]
    style NOTE fill:#f5f5f5,stroke:#999,color:#333
```

---

## 5. executor.Pool Internal Flow

Once work reaches the pool, the internal path is:

```mermaid
flowchart LR
    SUBMIT["Pool.Submit(task)\nor Pool.SubmitResume(task, exec)"]
    QUEUE["workCh buffered channel\n(FIFO, no priority ordering)"]
    DISPATCH["dispatch() goroutine\nwaits for semaphore slot\nchecks budget gate\nchecks depends_on satisfied\nresolves role → Type/Model (role-typed tasks)"]
    RUNNER["ContainerRunner\n(Docker; git clone; claude -p)\nor NativeRunner (provider.Provider tool-use loop)"]
    RESULT["handleRunResult\nwrite execution row\nupdate task state\nbroadcast WebSocket event\ncascade-fail dependents if terminal-fail\nmaybeUnblockParent if nested task completes"]

    SUBMIT --> QUEUE --> DISPATCH --> RUNNER --> RESULT
    RESULT -->|"READY or COMPLETED"| DONE2(["task observable\nvia WS / REST"])
    RESULT -->|"BLOCKED (ask_user)"| BLOCKED2(["workspace preserved\nin sandbox_dir;\nawait answer"])
    RESULT -->|"BLOCKED (subtasks)"| SUBTASK(["parent BLOCKED;\nPENDING children\nneed separate /run"])
    RESULT -->|"FAILED etc."| FAILED2(["Scheduler picks up\non next poll tick"])
```

---

## Notes

- **`propose_epic` / `propose_role_config`** write to storage only — no tasks
  are created and no pool call is made. They exist to let a running agent record
  planning-layer artifacts as side-effects of its execution.

- **`spawn_subtask` does not auto-submit.** The child task is created PENDING.
  The parent task's execution then ends, `handleRunResult` detects the pending
  children and moves the parent to BLOCKED. Something external must later call
  `POST /api/tasks/{id}/run` (or `claudomator start`) for each PENDING child.
  The `scripts/start-next-task` helper automates this; `StoryOrchestrator` does
  it automatically for story-pipeline role tasks via `spawnRoleTask`.

- **GitHub webhook does not auto-run.** `createCIFailureTask` stores the task
  at PENDING and returns the task ID. A human, chatbot, or polling script must
  explicitly call `POST /api/tasks/{id}/run` to queue it.

- **`story.RootTaskID` is immutable** after story creation. When a builder node
  is rejected and replaced, `task.CurrentAttempt(store, anchorID)` walks the
  `depends_on` chain forward to find the current live node. Every orchestrator
  caller resolves through this before acting.

- **Scheduler vs. StoryOrchestrator**: the `Scheduler` handles generic
  role-typed task lifecycle (FAILED → retry/escalate; BLOCKED → timeout
  resume). The `StoryOrchestrator` handles story-specific pipeline orchestration
  (Builder → Evaluators → Arbitration). They run in the same process but poll
  independently and write to separate concerns.