summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/workflow-and-entry-points.md137
1 files changed, 78 insertions, 59 deletions
diff --git a/docs/workflow-and-entry-points.md b/docs/workflow-and-entry-points.md
index d9312c5..367c5ba 100644
--- a/docs/workflow-and-entry-points.md
+++ b/docs/workflow-and-entry-points.md
@@ -8,41 +8,41 @@ arbitrated-review mechanism sits relative to that state machine.
## 1. Entry Points Overview
-Every arrow below eventually reaches `executor.Pool`, the single choke-point
-that bounds concurrency and gates spend.
+Every path that ultimately executes work passes through `executor.Pool`, the single
+choke-point that bounds concurrency and gates spend. Some entry points submit
+directly; others only write storage and let a poll loop or a human /run call submit.
```mermaid
flowchart TD
subgraph CLI["CLI (claudomator run)"]
- CLI_RUN["claudomator run <file.yaml>\n(internal/cli/run.go)"]
+ CLI_RUN["claudomator run <file.yaml>\n(internal/cli/run.go)\nParses YAML, creates + submits directly"]
end
subgraph REST["REST API"]
- API_CREATE["POST /api/tasks\n(creates PENDING task)"]
- API_RUN["POST /api/tasks/{id}/run\n(submits PENDING → QUEUED)"]
+ API_CREATE["POST /api/tasks\n(creates PENDING task in storage only)"]
+ API_RUN["POST /api/tasks/{id}/run\n(PENDING → QUEUED + Pool.Submit)"]
API_CREATE --> API_RUN
end
subgraph CHATBOT["Chatbot MCP (/chatbot/mcp)"]
- CB_SUBMIT["submit_task tool\n(internal/api/chatbotmcp.go)"]
- CB_STORY["create_story tool\n(creates Story row)"]
- CB_ACCEPT["accept_story tool\n(REVIEW_READY → DONE)"]
- CB_LIST["list_stories / get_story tools\n(read-only)"]
+ CB_SUBMIT["submit_task tool\n(creates + submits in one step)"]
+ CB_STORY["create_story tool\n(creates Story row in storage only;\nStoryOrchestrator spawns Builder on next tick)"]
+ CB_ACCEPT["accept_story tool\n(REVIEW_READY → DONE;\norchestrator triggers retro next tick)"]
end
subgraph WEBHOOK["GitHub Webhook"]
- WH["POST /api/webhooks/github\ncheck_run / workflow_run failure\n(internal/api/webhook.go)"]
+ WH["POST /api/webhooks/github\ncheck_run / workflow_run failure\n(internal/api/webhook.go)\ncreates PENDING task in storage;\ndoes NOT auto-submit to pool"]
end
subgraph STORY_API["Story REST API"]
- ST_CREATE["POST /api/stories\n(creates Story row)"]
+ ST_CREATE["POST /api/stories\n(creates Story row in storage only)"]
ST_ACCEPT["POST /api/stories/{id}/accept\n(REVIEW_READY → DONE)"]
end
subgraph AGENT_TOOLS["Agent Back-channel Tools\n(MCP /mcp or native tool-use loop)"]
- AG_SPAWN["spawn_subtask\n(no role= → child with parent_task_id)\n(role= → sibling with depends_on)"]
- AG_EPIC["propose_epic\n(creates/updates Epic, groups Stories)"]
- AG_RC["propose_role_config\n(creates draft role_configs version)"]
+ AG_SPAWN["spawn_subtask\n(creates PENDING child task;\nparent_task_id always set;\ndoes NOT auto-submit to pool)"]
+ AG_EPIC["propose_epic\n(creates/updates Epic row;\ngroups Stories; storage only)"]
+ AG_RC["propose_role_config\n(creates draft role_configs row;\nstorage only)"]
end
subgraph ORCHESTRATOR["StoryOrchestrator (poll, 15 s)\n(internal/scheduler/story_orchestrator.go)"]
@@ -57,32 +57,34 @@ flowchart TD
SC_TIMEOUT["tickAskUserTimeouts\n(auto-answer + resume BLOCKED tasks\noutstanding > ask_user_timeout)"]
end
+ STORAGE[("SQLite storage\n(tasks table)")]
POOL(["executor.Pool\n(bounded dispatcher)"])
- %% CLI path
- CLI_RUN -->|"Pool.Submit() directly"| POOL
+ %% CLI path — direct submit
+ CLI_RUN -->|"creates + QUEUED + Pool.Submit()"| POOL
- %% REST path
+ %% REST path — two-step
API_RUN -->|"Pool.Submit()"| POOL
- %% Chatbot MCP paths
- CB_SUBMIT -->|"creates task + auto-submits"| POOL
- CB_STORY -->|"story row only;\nStoryOrchestrator spawns Builder"| ORCHESTRATOR
- CB_ACCEPT -->|"story REVIEW_READY → DONE;\ntriggers processRetro next tick"| ORCHESTRATOR
+ %% Chatbot MCP — submit_task is direct; story tools are storage-only
+ CB_SUBMIT -->|"creates + QUEUED + Pool.Submit()"| POOL
+ CB_STORY -->|"Story row only"| STORAGE
+ CB_ACCEPT -->|"Story REVIEW_READY → DONE"| STORAGE
- %% Webhook path
- WH -->|"creates task + auto-runs"| POOL
+ %% Webhook — creates PENDING task only; needs /run to execute
+ WH -->|"creates PENDING task"| STORAGE
- %% Story REST paths
- ST_CREATE -->|"story row only"| ORCHESTRATOR
- ST_ACCEPT -->|"REVIEW_READY → DONE"| ORCHESTRATOR
+ %% Story REST — storage-only; orchestrator picks up on next tick
+ ST_CREATE -->|"Story row only"| STORAGE
+ ST_ACCEPT -->|"Story REVIEW_READY → DONE"| STORAGE
- %% Agent tool paths
- AG_SPAWN -->|"Pool.Submit() for new child/sibling"| POOL
- AG_EPIC -->|"storage only\n(no task spawned)"| POOL
- AG_RC -->|"storage only\n(no task spawned)"| POOL
+ %% Agent tools — spawn_subtask is storage-only; propose_epic/role_config are pure storage
+ AG_SPAWN -->|"creates PENDING child task\n(scripts/start-next-task,\nStoryOrchestrator, or /run\nsubmit it later)"| STORAGE
+ AG_EPIC -->|"Epic/Story rows only"| STORAGE
+ AG_RC -->|"role_configs draft row only"| STORAGE
- %% StoryOrchestrator → Pool
+ %% StoryOrchestrator spawns tasks AND submits them
+ STORAGE -->|"StoryOrchestrator polls stories;\nspawnRoleTask: CreateTask + QUEUED + Pool.Submit()"| SO_EVAL
SO_EVAL -->|"Store.CreateTask + Pool.Submit()"| POOL
SO_ARB -->|"Store.CreateTask + Pool.Submit()"| POOL
SO_FIX -->|"Store.CreateTask + Pool.Submit()"| POOL
@@ -92,31 +94,38 @@ flowchart TD
SC_FAIL -->|"Pool.Submit() or Pool.SubmitResume()"| POOL
SC_TIMEOUT -->|"Pool.SubmitResume()"| POOL
- %% Orchestrator watches Pool results
- POOL -->|"task reaches READY/COMPLETED/FAILED;\norchestrator polls"| ORCHESTRATOR
+ %% Pool outcomes feed the schedulers
+ POOL -->|"task reaches READY/COMPLETED/FAILED;\norchestrator polls storage"| ORCHESTRATOR
POOL -->|"task reaches FAILED;\nscheduler polls"| SCHEDULER
- POOL -->|"task stays BLOCKED;\nscheduler polls"| SCHEDULER
+ POOL -->|"task stays BLOCKED;\nscheduler polls (ask_user timeout)"| SCHEDULER
```
+> **Note on PENDING subtasks:** When an agent calls `spawn_subtask`, the child task
+> lands in PENDING state in storage. Nothing auto-submits it. Dispatch happens via one
+> of: (a) the `scripts/start-next-task` helper (calls `POST /api/tasks/{id}/run`),
+> (b) `StoryOrchestrator.spawnRoleTask` if the subtask is a story-pipeline role task,
+> (c) a human or chatbot calling `POST /api/tasks/{id}/run`, or
+> (d) `RecoverStaleQueued` on server restart for tasks already in QUEUED state.
+
---
## 2. Detailed Entry Point Reference
| Entry Point | Code Location | What It Creates | Who Submits to Pool |
|---|---|---|---|
-| `claudomator run <file>` | `internal/cli/run.go` | Task(s) from YAML | `cli` directly via `Pool.Submit` |
-| `POST /api/tasks` + `POST /api/tasks/{id}/run` | `internal/api/taskops.go` | PENDING task, then QUEUED | `api.Server` via `Pool.Submit` |
-| Chatbot MCP `submit_task` | `internal/api/chatbotmcp.go` | Task, immediately submitted | `chatbotmcp` via `Pool.Submit` |
-| Chatbot MCP `create_story` | `internal/api/chatbotmcp.go` | Story row | `StoryOrchestrator` (next tick) |
-| Chatbot MCP `accept_story` | `internal/api/chatbotmcp.go` | Story status DONE | `StoryOrchestrator` (triggers retro) |
-| `POST /api/stories` | `internal/api/stories.go` | Story row | `StoryOrchestrator` (next tick) |
-| `POST /api/stories/{id}/accept` | `internal/api/stories.go` | Story status DONE | `StoryOrchestrator` (triggers retro) |
-| `POST /api/webhooks/github` | `internal/api/webhook.go` | Task auto-tagged `["ci","auto"]` | `api.Server` via `Pool.Submit` |
-| Agent `spawn_subtask` (no `role=`) | `internal/executor/agentmcp.go`, `internal/agentloop/tools.go` | Child task with `parent_task_id` | `Pool.Submit` |
-| Agent `spawn_subtask` (with `role=`) | same | Sibling task with `depends_on`, `Agent.Role` set, no Type/Model | `Pool.Submit`; role resolved at dispatch |
-| Agent `propose_epic` | same | Epic row; groups existing stories | storage only, no task |
-| Agent `propose_role_config` | same | Draft `role_configs` version | storage only, no task |
-| `StoryOrchestrator.ensureEvaluators` | `internal/scheduler/story_orchestrator.go` | 4 Evaluator tasks per builder node | `Store.CreateTask` + `Pool.Submit` |
+| `claudomator run <file>` | `internal/cli/run.go` | Task(s) from YAML | `cli` directly: `CreateTask` → `UpdateTaskState(QUEUED)` → `Pool.Submit` |
+| `POST /api/tasks` + `POST /api/tasks/{id}/run` | `internal/api/server.go`, `taskops.go` | PENDING task, then QUEUED | `handleRunTask` via `Pool.Submit` |
+| Chatbot MCP `submit_task` | `internal/api/chatbotmcp.go` | Task (PENDING→QUEUED in one step) | `submitTask()` via `Pool.Submit` |
+| Chatbot MCP `create_story` | `internal/api/chatbotmcp.go` | Story row in storage | `StoryOrchestrator` (next tick): creates + submits Builder task |
+| Chatbot MCP `accept_story` | `internal/api/chatbotmcp.go` | Story status DONE in storage | `StoryOrchestrator` (next tick): spawns retro task |
+| `POST /api/stories` | `internal/api/stories.go` | Story row in storage | `StoryOrchestrator` (next tick) |
+| `POST /api/stories/{id}/accept` | `internal/api/stories.go` | Story status DONE in storage | `StoryOrchestrator` (next tick): spawns retro task |
+| `POST /api/webhooks/github` | `internal/api/webhook.go` | Task in **PENDING** state only | Human/chatbot/script must call `POST /api/tasks/{id}/run` |
+| Agent `spawn_subtask` (no `role=`) | `internal/executor/agentmcp.go`, `internal/agentloop/tools.go` | Child task with `parent_task_id`, `agent.type=claude` | **Not auto-submitted**; parent goes BLOCKED; child is dispatched later |
+| Agent `spawn_subtask` (with `role=`) | same | Child task with `parent_task_id`, `agent.role` set, no Type/Model | **Not auto-submitted**; role resolved at eventual dispatch via `Pool.execute()` |
+| Agent `propose_epic` | `internal/executor/agentmcp.go`, `internal/agentloop/tools.go` | Epic row; links existing stories | Storage only — no task created, no pool call |
+| Agent `propose_role_config` | same | Draft `role_configs` version | Storage only — no task created, no pool call |
+| `StoryOrchestrator.ensureEvaluators` | `internal/scheduler/story_orchestrator.go` | 4 Evaluator tasks per builder node | `spawnRoleTask`: `CreateTask` → `UpdateTaskState(QUEUED)` → `Pool.Submit` |
| `StoryOrchestrator.ensureArbitration` | same | 1 `planner`-role Arbitration task | same |
| `StoryOrchestrator.ensureFixAttempt` | same | New root builder task after rejection | same |
| `StoryOrchestrator.spawnNestedFixAttempt` | same | New nested builder task after rejection | same |
@@ -130,7 +139,7 @@ flowchart TD
```mermaid
stateDiagram-v2
- [*] --> PENDING : task created
+ [*] --> PENDING : task created\n(POST /api/tasks, webhook,\nspawn_subtask, story rows)
PENDING --> QUEUED : /run submitted\n(Pool.Submit)
@@ -206,13 +215,13 @@ flowchart TD
subgraph ReviewCycle["Review Cycle (per builder node, any depth)"]
EVAL["4 Evaluator tasks\nevaluator_quality\nevaluator_security\nevaluator_correctness\nevaluator_performance\n\n(depends_on: builder node)"]
ARB["1 Arbitration task\nrole: planner\n\n(depends_on: all 4 evaluators)"]
- APPROVE{"Approved?"}
+ APPROVE{"Approved?\n(report_verdict event;\ndefault: approve)"}
FIX["Fix-attempt task\n(same role, new builder;\ndepends_on: rejected node)"]
end
subgraph Terminal["Terminal states"]
COMP["builder node → COMPLETED\n(if root: story → REVIEW_READY)"]
- HUMAN["POST /api/stories/{id}/accept\n(REVIEW_READY → DONE)"]
+ HUMAN["POST /api/stories/{id}/accept\n(REVIEW_READY → DONE)\n[only manual step in the chain]"]
RETRO["processRetro: retro-role task\nreflects, proposes role_configs drafts"]
DONE["story → DONE"]
end
@@ -241,13 +250,14 @@ Once work reaches the pool, the internal path is:
flowchart LR
SUBMIT["Pool.Submit(task)\nor Pool.SubmitResume(task, exec)"]
QUEUE["workCh buffered channel\n(FIFO, no priority ordering)"]
- DISPATCH["dispatch() goroutine\nwaits for semaphore slot\nchecks budget gate\nchecks depends_on satisfied"]
- RUNNER["ContainerRunner\n(Docker; git clone; claude -p)\nor LocalRunner (OpenAI-compatible HTTP)"]
- RESULT["handleRunResult\nwrite execution row\nupdate task state\nbroadcast WebSocket event\ncascade-fail dependents if terminal-fail"]
+ DISPATCH["dispatch() goroutine\nwaits for semaphore slot\nchecks budget gate\nchecks depends_on satisfied\nresolves role → Type/Model (role-typed tasks)"]
+ RUNNER["ContainerRunner\n(Docker; git clone; claude -p)\nor NativeRunner (provider.Provider tool-use loop)"]
+ RESULT["handleRunResult\nwrite execution row\nupdate task state\nbroadcast WebSocket event\ncascade-fail dependents if terminal-fail\nmaybeUnblockParent if nested task completes"]
SUBMIT --> QUEUE --> DISPATCH --> RUNNER --> RESULT
RESULT -->|"READY or COMPLETED"| DONE2(["task observable\nvia WS / REST"])
- RESULT -->|"BLOCKED"| BLOCKED2(["workspace preserved\nin sandbox_dir;\nawait answer"])
+ RESULT -->|"BLOCKED (ask_user)"| BLOCKED2(["workspace preserved\nin sandbox_dir;\nawait answer"])
+ RESULT -->|"BLOCKED (subtasks)"| SUBTASK(["parent BLOCKED;\nPENDING children\nneed separate /run"])
RESULT -->|"FAILED etc."| FAILED2(["Scheduler picks up\non next poll tick"])
```
@@ -255,11 +265,20 @@ flowchart LR
## Notes
-- **`propose_epic` / `propose_role_config`** are agent tools that write to
- storage only — they do not create tasks or trigger any further scheduling.
- They exist to let a running agent record planning-layer artifacts (epic
- groupings; improved role system prompts for human review) as a side-effect of
- its execution.
+- **`propose_epic` / `propose_role_config`** write to storage only — no tasks
+ are created and no pool call is made. They exist to let a running agent record
+ planning-layer artifacts as side-effects of its execution.
+
+- **`spawn_subtask` does not auto-submit.** The child task is created PENDING.
+ The parent task's execution then ends, `handleRunResult` detects the pending
+ children and moves the parent to BLOCKED. Something external must later call
+ `POST /api/tasks/{id}/run` (or `claudomator start`) for each PENDING child.
+ The `scripts/start-next-task` helper automates this; `StoryOrchestrator` does
+ it automatically for story-pipeline role tasks via `spawnRoleTask`.
+
+- **GitHub webhook does not auto-run.** `createCIFailureTask` stores the task
+ at PENDING and returns the task ID. A human, chatbot, or polling script must
+ explicitly call `POST /api/tasks/{id}/run` to queue it.
- **`story.RootTaskID` is immutable** after story creation. When a builder node
is rejected and replaced, `task.CurrentAttempt(store, anchorID)` walks the