diff options
| author | Peter Stone <thepeterstone@gmail.com> | 2026-03-21 21:23:42 +0000 |
|---|---|---|
| committer | Peter Stone <thepeterstone@gmail.com> | 2026-03-21 21:23:42 +0000 |
| commit | 888f3014b42ff48f597d0a81e9f52104d19be6db (patch) | |
| tree | 133d1c2e45affe293624991c3b8239b2429c21e9 | |
| parent | a10e7478a130d6453abbd8fb0694948785dd2155 (diff) | |
feat: Phase 2 — project registry, legacy field cleanup, credential path fix
- task.Project type + storage CRUD + UpsertProject + SeedProjects
- Remove AgentConfig.ProjectDir, RepositoryURL, SkipPlanning
- Remove ContainerRunner fallback git init logic
- Project API endpoints: GET/POST /api/projects, GET/PUT /api/projects/{id}
- processResult no longer extracts changestats (pool-side only)
- claude_config_dir config field; default to credentials/claude/
- New scripts: sync-credentials, fix-permissions, check-token
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
| -rw-r--r-- | CLAUDE.md | 14 | ||||
| -rw-r--r-- | SESSION_STATE.md | 86 | ||||
| -rw-r--r-- | docs/adr/007-planning-layer-and-story-model.md | 6 | ||||
| -rw-r--r-- | internal/api/deployment.go | 4 | ||||
| -rw-r--r-- | internal/api/elaborate.go | 12 | ||||
| -rw-r--r-- | internal/api/projects.go | 71 | ||||
| -rw-r--r-- | internal/api/server.go | 17 | ||||
| -rw-r--r-- | internal/api/server_test.go | 72 | ||||
| -rw-r--r-- | internal/api/task_view.go | 4 | ||||
| -rw-r--r-- | internal/api/webhook_test.go | 8 | ||||
| -rw-r--r-- | internal/executor/container.go | 37 | ||||
| -rw-r--r-- | internal/storage/db.go | 71 | ||||
| -rw-r--r-- | internal/storage/db_test.go | 66 | ||||
| -rw-r--r-- | internal/storage/seed.go | 46 | ||||
| -rw-r--r-- | internal/task/project.go | 11 | ||||
| -rw-r--r-- | internal/task/task.go | 3 | ||||
| -rw-r--r-- | internal/task/validator_test.go | 1 | ||||
| -rw-r--r-- | scripts/check-token | 78 | ||||
| -rw-r--r-- | scripts/fix-permissions | 43 | ||||
| -rw-r--r-- | scripts/sync-credentials | 40 |
20 files changed, 571 insertions, 119 deletions
@@ -166,6 +166,20 @@ A task is created for: Tasks are tagged `["ci", "auto"]`, capped at $3 USD, and use tools: Read, Edit, Bash, Glob, Grep. +## System Maintenance (Cron) + +The following crontab entries are required for system operation and must be maintained for the root user: + +```crontab +# Sync Claude and Gemini credentials every 10 minutes +*/10 * * * * /workspace/claudomator/scripts/sync-credentials + +# Start the next queued task every 20 minutes +*/20 * * * * /workspace/claudomator/scripts/start-next-task >> /var/log/claudomator-cron.log 2>&1 +``` + +> **Note:** These requirements are critical for agent authentication and automated task progression. + ## Agent Tooling (`ct` CLI) Agents running inside containers have access to `ct`, a pre-built CLI for interacting with the Claudomator API. It is installed at `/usr/local/bin/ct` in the container image. **Use `ct` to create and manage subtasks — do not attempt raw `curl` API calls.** diff --git a/SESSION_STATE.md b/SESSION_STATE.md index 6d599f1..6fb8033 100644 --- a/SESSION_STATE.md +++ b/SESSION_STATE.md @@ -1,9 +1,9 @@ # SESSION_STATE.md ## Current Task Goal -Add `project` field to tasks — allow tasks to carry a project label displayed in the UI. +ADR-007 implementation: Epic→Story→Task→Subtask hierarchy, project registry, Doot integration -## Status: COMPLETED +## Status: IN_PROGRESS --- @@ -11,50 +11,62 @@ Add `project` field to tasks — allow tasks to carry a project label displayed | Step | Description | Test / Verification | |------|-------------|---------------------| -| 1 | Add `Project` field to `Task` struct and YAML parsing (`internal/task/task.go:77`) | `TestTask_ParseFile` (existing, passing) | -| 2 | Add `project` column to SQLite schema and all CRUD queries (`internal/storage/db.go`) | `TestDB_*` suite in `internal/storage` — all passing | -| 3 | Expose `project` in REST API create/list/get endpoints (`internal/api/`) | `internal/api` suite — all passing | -| 4 | Expose `project` in CLI (`internal/cli/`) | `internal/cli` suite — all passing | -| 5 | Display `project` badge on task cards in web UI (`web/app.js:159-163`) | Visual; `grep -n project web/app.js` confirms presence | - -### Commits -``` -054ec8b feat: add Project field to Task struct and YAML parsing -072652f feat: add project column to storage -26dc313 feat: expose project field in API and CLI -b838150 feat: display project field in web UI -``` +| Phase 1 | Doot dead code removal: Bug struct, BugToAtom, bug store methods, bug handlers, bug routes, bugs.html template, TypeNote, AddMealToPlanner stub | `go test ./...` in /workspace/doot — all pass (2 pre-existing failures unrelated) | +| Phase 2 | Claudomator project registry: `task.Project` type, storage CRUD + UpsertProject, seed.go, API endpoints (GET/POST /api/projects, GET/PUT /api/projects/{id}), legacy AgentConfig.ProjectDir/RepositoryURL/SkipPlanning fields removed, container.go fallback removed, fallbackGitInit removed, processResult changestats extraction removed (pool-side only) | `TestCreateProject`, `TestListProjects`, `TestUpdateProject`, `TestProjects_CRUD` — all pass | --- -## Test Suite Results (2026-03-16) +## Next Steps (Claudomator tasks created) -``` -ok internal/api 5.664s -ok internal/cli 6.136s -ok internal/config 1.023s -ok internal/notify 1.024s -ok internal/reporter 1.017s -ok internal/storage 1.421s -ok internal/task 1.022s -FAIL internal/executor 1.391s ← PRE-EXISTING failures (unrelated) -``` +Phases 3–6 are queued as Claudomator tasks. See `ct task list` or the web UI. -### Pre-existing failures in `internal/executor` (not caused by this feature) +| Task ID | Phase | Status | Depends On | +|---------|-------|--------|------------| +| f8829d6f-b8b6-4ff2-9c1a-e55dd3ab300e | Phase 3: Stories data model | PENDING | — | +| c8a0dc6c-0605-4acb-a789-1155ad8824cb | Phase 4: Story execution and deploy | PENDING | Phase 3 | +| faf5a371-8f1c-46a3-bb74-b0df1f062dee | Phase 5: Story elaboration | PENDING | Phase 3 | +| f39af70f-72c5-4ac1-9522-83c2e11b37c9 | Phase 6: Doot — Claudomator integration | PENDING | Phase 3 | -- `TestTeardownSandbox_AutocommitsChanges` — `git log in bare repo: exit status 128` -- `TestTeardownSandbox_BuildSuccess_ProceedsToAutocommit` — `git log in bare repo: exit status 128` -- `TestPool_RecoverStaleRunning` — `execution status: want FAILED, got [...]` +Instruction files: `scripts/.claude/phase{3,4,5,6}-*-instructions.txt` -These fail on commits predating the project field feature (verified by running tests against `local/master`). +### Phase 3: Stories data model (claudomator repo) +- `internal/task/story.go` — Story struct + ValidStoryTransition +- `internal/storage/db.go` — stories table + story_id on tasks, CRUD + ListTasksByStory +- `internal/api/stories.go` — story API endpoints +- Tests: ValidStoryTransition, CRUD, depends_on auto-wire ---- +### Phase 4: Story execution and deploy (claudomator repo, depends Phase 3) +- `internal/executor/executor.go` — checkStoryCompletion → SHIPPABLE +- `internal/executor/container.go` — checkout story branch after clone +- `internal/api/stories.go` — POST /api/stories/{id}/branch + +### Phase 5: Story elaboration (claudomator repo, depends Phase 3) +- `internal/api/elaborate.go` — POST /api/stories/elaborate + approve +- SeedProjects called at server startup -## Next Steps -None — feature is complete. Pre-existing executor failures are tracked separately. +### Phase 6: Doot — Claudomator integration (doot repo, depends Phase 3) +- `internal/api/claudomator.go` — ClaudomatorClient +- `internal/models/atom.go` — StoryToAtom, SourceClaudomator +- `internal/handlers/atoms.go` — BuildUnifiedAtomList extended +- `cmd/dashboard/main.go` — wire ClaudomatorURL config --- -## Process Notes -- Schema migration uses `ALTER TABLE tasks ADD COLUMN project TEXT` (idempotent via existing migration guard). -- `scanTask` uses `sql.NullString` for backward compatibility with existing rows that have no project value. +## Key Files Changed (Phases 1-2) + +### Claudomator +- `internal/task/project.go` — new Project struct +- `internal/task/task.go` — removed Agent.ProjectDir, Agent.RepositoryURL, Agent.SkipPlanning +- `internal/storage/db.go` — projects table migration + CRUD +- `internal/storage/seed.go` — SeedProjects upserts claudomator + nav on startup +- `internal/api/projects.go` — project CRUD handlers +- `internal/api/server.go` — project routes; processResult no longer extracts changestats +- `internal/api/deployment.go` + `task_view.go` — use tk.RepositoryURL (was tk.Agent.ProjectDir) +- `internal/executor/container.go` — fallback logic removed; requires t.RepositoryURL + +### Doot +- Bug feature removed entirely (models, handlers, store, routes, template, migration) +- `migrations/018_drop_bugs.sql` — DROP TABLE IF EXISTS bugs +- `internal/api/interfaces.go` — AddMealToPlanner removed from PlanToEatAPI +- `internal/api/plantoeat.go` — AddMealToPlanner stub removed +- `internal/models/atom.go` — SourceBug, TypeBug, TypeNote, BugToAtom removed diff --git a/docs/adr/007-planning-layer-and-story-model.md b/docs/adr/007-planning-layer-and-story-model.md index 2ca2afd..7efb66d 100644 --- a/docs/adr/007-planning-layer-and-story-model.md +++ b/docs/adr/007-planning-layer-and-story-model.md @@ -74,6 +74,7 @@ Each story has a dedicated Git branch. Subtasks execute sequentially, each cloni - **Ordered execution enforced** — subtasks run strictly in order; each depends on the previous commit - **Reviewable history** — the story branch accumulates one commit per subtask, giving a clean, auditable record before merge - **Clear recovery points** — if subtask N fails, roll back to subtask N-1's commit, fix the subtask definition, rerun +- **Resilient to transient API failures** — transient rate-limit errors (429) do not fail the story or subtask; the executor requeues the task and "pauses" the story until the agent is unblocked, preserving the sequential chain. **Tradeoffs accepted:** - Clone and container creation cost is paid per subtask (not amortized across the story). Acceptable at current usage scale. @@ -162,6 +163,8 @@ The elaborator selects `type` based on change scope. Curl is the default for sma - Evidence (response bodies, test output excerpts, screenshots for playwright) - Overall verdict: pass → story moves to `REVIEW_READY`; fail → story moves to `NEEDS_FIX` with report attached +Validation subtasks are governed by the same pool-level rate-limit resilience; a 429 during validation will requeue the subtask rather than failing the story. + #### Failure Recovery If a subtask fails mid-story: pause the story and require human review before resuming. The options at that point are: @@ -188,6 +191,8 @@ Policy beyond this is deferred until failure patterns are observed in practice. - Elaborator output extended: `validation` block (type, steps, success_criteria) stored as `validation_json` on story - Remove `Agent.RepositoryURL`, `Agent.ProjectDir` legacy fields, `skip_planning`, `fallbackGitInit()` - Remove duplicate changestats extraction (keep pool-side, remove API server-side) +- Pool-level "requeue-and-skip" logic for rate-limited agents: tasks return to `QUEUED` and release worker slots if all candidate agents are blocked, allowing the system to "wait out" 429 errors without failing stories. +- Background "Recovery Scheduler" goroutine: periodically (every 30m or as hinted by API) runs minimal "test tasks" to verify agent availability and unblock the pool. **Doot changes:** - New `SourceClaudomator` atom source @@ -224,6 +229,7 @@ Story creation is driven by a beefed-up version of Claudomator's existing elabor - A `git fetch` (not pull) at elaboration start updates remote refs without touching the working tree - Branch creation is deferred to approval — elaboration agent is purely read-only - Execution clones use `git clone --reference /local/path <remote>` — reuses local object store, fetches only the delta; significantly faster than cold clone +- Rate-limit aware — if the elaboration agent is blocked, the UI surfaces the status and resumes automatically once unblocked via the Recovery Scheduler. ### Project Registry diff --git a/internal/api/deployment.go b/internal/api/deployment.go index d927545..8972fe2 100644 --- a/internal/api/deployment.go +++ b/internal/api/deployment.go @@ -23,7 +23,7 @@ func (s *Server) handleGetDeploymentStatus(w http.ResponseWriter, r *http.Reques if err != nil { if err == sql.ErrNoRows { // No execution yet — return status with no fix commits. - status := deployment.Check(nil, tk.Agent.ProjectDir) + status := deployment.Check(nil, tk.RepositoryURL) writeJSON(w, http.StatusOK, status) return } @@ -31,6 +31,6 @@ func (s *Server) handleGetDeploymentStatus(w http.ResponseWriter, r *http.Reques return } - status := deployment.Check(exec.Commits, tk.Agent.ProjectDir) + status := deployment.Check(exec.Commits, tk.RepositoryURL) writeJSON(w, http.StatusOK, status) } diff --git a/internal/api/elaborate.go b/internal/api/elaborate.go index 0c681ae..2c164d3 100644 --- a/internal/api/elaborate.go +++ b/internal/api/elaborate.go @@ -281,7 +281,9 @@ func (s *Server) handleElaborateTask(w http.ResponseWriter, r *http.Request) { } var input struct { - Prompt string `json:"prompt"` + Prompt string `json:"prompt"` + ProjectID string `json:"project_id"` + // project_dir kept for backward compat; project_id takes precedence ProjectDir string `json:"project_dir"` } if err := json.NewDecoder(r.Body).Decode(&input); err != nil { @@ -294,11 +296,15 @@ func (s *Server) handleElaborateTask(w http.ResponseWriter, r *http.Request) { } workDir := s.workDir - if input.ProjectDir != "" { + if input.ProjectID != "" { + if proj, err := s.store.GetProject(input.ProjectID); err == nil { + workDir = proj.LocalPath + } + } else if input.ProjectDir != "" { workDir = input.ProjectDir } - if input.ProjectDir != "" { + if workDir != s.workDir { go s.appendRawNarrative(workDir, input.Prompt) } diff --git a/internal/api/projects.go b/internal/api/projects.go new file mode 100644 index 0000000..d3dbbf9 --- /dev/null +++ b/internal/api/projects.go @@ -0,0 +1,71 @@ +package api + +import ( + "encoding/json" + "net/http" + + "github.com/google/uuid" + "github.com/thepeterstone/claudomator/internal/task" +) + +func (s *Server) handleListProjects(w http.ResponseWriter, r *http.Request) { + projects, err := s.store.ListProjects() + if err != nil { + writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()}) + return + } + if projects == nil { + projects = []*task.Project{} + } + writeJSON(w, http.StatusOK, projects) +} + +func (s *Server) handleCreateProject(w http.ResponseWriter, r *http.Request) { + var p task.Project + if err := json.NewDecoder(r.Body).Decode(&p); err != nil { + writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid JSON: " + err.Error()}) + return + } + if p.Name == "" { + writeJSON(w, http.StatusBadRequest, map[string]string{"error": "name is required"}) + return + } + if p.ID == "" { + p.ID = uuid.New().String() + } + if err := s.store.CreateProject(&p); err != nil { + writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()}) + return + } + writeJSON(w, http.StatusCreated, p) +} + +func (s *Server) handleGetProject(w http.ResponseWriter, r *http.Request) { + id := r.PathValue("id") + p, err := s.store.GetProject(id) + if err != nil { + writeJSON(w, http.StatusNotFound, map[string]string{"error": "project not found"}) + return + } + writeJSON(w, http.StatusOK, p) +} + +func (s *Server) handleUpdateProject(w http.ResponseWriter, r *http.Request) { + id := r.PathValue("id") + existing, err := s.store.GetProject(id) + if err != nil { + writeJSON(w, http.StatusNotFound, map[string]string{"error": "project not found"}) + return + } + if err := json.NewDecoder(r.Body).Decode(existing); err != nil { + writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid JSON: " + err.Error()}) + return + } + existing.ID = id // ensure ID cannot be changed via body + if err := s.store.UpdateProject(existing); err != nil { + writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()}) + return + } + writeJSON(w, http.StatusOK, existing) +} + diff --git a/internal/api/server.go b/internal/api/server.go index 0127ab9..65823b4 100644 --- a/internal/api/server.go +++ b/internal/api/server.go @@ -133,6 +133,10 @@ func (s *Server) routes() { s.mux.HandleFunc("GET /api/ws", s.handleWebSocket) s.mux.HandleFunc("GET /api/workspaces", s.handleListWorkspaces) s.mux.HandleFunc("GET /api/tasks/{id}/deployment-status", s.handleGetDeploymentStatus) + s.mux.HandleFunc("GET /api/projects", s.handleListProjects) + s.mux.HandleFunc("POST /api/projects", s.handleCreateProject) + s.mux.HandleFunc("GET /api/projects/{id}", s.handleGetProject) + s.mux.HandleFunc("PUT /api/projects/{id}", s.handleUpdateProject) s.mux.HandleFunc("GET /api/health", s.handleHealth) s.mux.HandleFunc("POST /api/webhooks/github", s.handleGitHubWebhook) s.mux.HandleFunc("GET /api/push/vapid-key", s.handleGetVAPIDKey) @@ -153,16 +157,7 @@ func (s *Server) forwardResults() { } // processResult broadcasts a task completion event via WebSocket and calls the notifier if set. -// It also parses git diff stats from the execution stdout log and persists them. func (s *Server) processResult(result *executor.Result) { - if result.Execution.StdoutPath != "" { - if stats := parseChangestatFromFile(result.Execution.StdoutPath); stats != nil { - if err := s.store.UpdateExecutionChangestats(result.Execution.ID, stats); err != nil { - s.logger.Error("failed to store changestats", "execID", result.Execution.ID, "error", err) - } - } - } - event := map[string]interface{}{ "type": "task_completed", "task_id": result.TaskID, @@ -463,10 +458,6 @@ func (s *Server) handleCreateTask(w http.ResponseWriter, r *http.Request) { ParentTaskID: input.ParentTaskID, } - // Fallback for repository_url if only provided in Agent config - if t.RepositoryURL == "" && input.Agent.ProjectDir != "" { - t.RepositoryURL = input.Agent.ProjectDir - } if t.Agent.Type == "" { t.Agent.Type = "claude" } diff --git a/internal/api/server_test.go b/internal/api/server_test.go index 8ff4227..27fc645 100644 --- a/internal/api/server_test.go +++ b/internal/api/server_test.go @@ -1641,34 +1641,31 @@ func TestRunTask_AgentCancelled_TaskSetToCancelled(t *testing.T) { } } -// TestGetTask_IncludesChangestats verifies that after processResult parses git diff stats -// from the execution stdout log, they appear in the execution history response. +// TestGetTask_IncludesChangestats verifies that changestats stored on an execution +// are returned correctly by GET /api/tasks/{id}/executions. func TestGetTask_IncludesChangestats(t *testing.T) { srv, store := testServer(t) tk := createTaskWithState(t, store, "cs-task-1", task.StateCompleted) - // Write a stdout log with a git diff --stat summary line. - dir := t.TempDir() - stdoutPath := filepath.Join(dir, "stdout.log") - logContent := "Agent output line 1\n3 files changed, 50 insertions(+), 10 deletions(-)\nAgent output line 2\n" - if err := os.WriteFile(stdoutPath, []byte(logContent), 0600); err != nil { - t.Fatal(err) - } - exec := &storage.Execution{ - ID: "cs-exec-1", - TaskID: tk.ID, - StartTime: time.Now().UTC(), - EndTime: time.Now().UTC().Add(time.Minute), - Status: "COMPLETED", - StdoutPath: stdoutPath, + ID: "cs-exec-1", + TaskID: tk.ID, + StartTime: time.Now().UTC(), + EndTime: time.Now().UTC().Add(time.Minute), + Status: "COMPLETED", } if err := store.CreateExecution(exec); err != nil { t.Fatal(err) } - // processResult should parse changestats from the stdout log and store them. + // Pool stores changestats after execution; simulate by calling UpdateExecutionChangestats directly. + cs := &task.Changestats{FilesChanged: 3, LinesAdded: 50, LinesRemoved: 10} + if err := store.UpdateExecutionChangestats(exec.ID, cs); err != nil { + t.Fatal(err) + } + + // processResult broadcasts but does NOT parse changestats (that's the pool's job). result := &executor.Result{ TaskID: tk.ID, Execution: exec, @@ -1976,3 +1973,44 @@ func TestListTasks_NonReadyTask_OmitsDeploymentStatus(t *testing.T) { t.Error("PENDING task should not include deployment_status field") } } + +func TestProjects_CRUD(t *testing.T) { + srv, _ := testServer(t) + + // Create + body := `{"name":"testproj","local_path":"/workspace/testproj","type":"web"}` + req := httptest.NewRequest("POST", "/api/projects", strings.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + w := httptest.NewRecorder() + srv.Handler().ServeHTTP(w, req) + if w.Code != http.StatusCreated { + t.Fatalf("POST /api/projects: want 201, got %d; body: %s", w.Code, w.Body.String()) + } + var created map[string]interface{} + json.NewDecoder(w.Body).Decode(&created) + id, _ := created["id"].(string) + if id == "" { + t.Fatal("created project has no id") + } + + // Get + req = httptest.NewRequest("GET", "/api/projects/"+id, nil) + w = httptest.NewRecorder() + srv.Handler().ServeHTTP(w, req) + if w.Code != http.StatusOK { + t.Fatalf("GET /api/projects/%s: want 200, got %d", id, w.Code) + } + + // List + req = httptest.NewRequest("GET", "/api/projects", nil) + w = httptest.NewRecorder() + srv.Handler().ServeHTTP(w, req) + if w.Code != http.StatusOK { + t.Fatalf("GET /api/projects: want 200, got %d", w.Code) + } + var list []interface{} + json.NewDecoder(w.Body).Decode(&list) + if len(list) == 0 { + t.Error("expected at least one project in list") + } +} diff --git a/internal/api/task_view.go b/internal/api/task_view.go index 5791058..e6e7097 100644 --- a/internal/api/task_view.go +++ b/internal/api/task_view.go @@ -30,12 +30,12 @@ func (s *Server) enrichTask(tk *task.Task) *taskView { if err != nil { if err == sql.ErrNoRows { // No execution yet — still include deployment status (empty commits). - view.DeploymentStatus = deployment.Check(nil, tk.Agent.ProjectDir) + view.DeploymentStatus = deployment.Check(nil, tk.RepositoryURL) } return view } view.Changestats = exec.Changestats - view.DeploymentStatus = deployment.Check(exec.Commits, tk.Agent.ProjectDir) + view.DeploymentStatus = deployment.Check(exec.Commits, tk.RepositoryURL) return view } diff --git a/internal/api/webhook_test.go b/internal/api/webhook_test.go index 0fc9664..967b62b 100644 --- a/internal/api/webhook_test.go +++ b/internal/api/webhook_test.go @@ -380,9 +380,9 @@ func TestGitHubWebhook_FallbackToSingleProject(t *testing.T) { } } -func TestGitHubWebhook_NoProjectsConfigured_CreatesTaskWithoutProjectDir(t *testing.T) { +func TestGitHubWebhook_NoProjectsConfigured_CreatesTaskWithGitHubURL(t *testing.T) { srv, store := testServer(t) - // No projects configured — task should still be created, just no project dir set. + // No projects configured — task should still be created with the GitHub remote URL. w := webhookPost(t, srv, "check_run", checkRunFailurePayload, "") @@ -395,8 +395,8 @@ func TestGitHubWebhook_NoProjectsConfigured_CreatesTaskWithoutProjectDir(t *test if err != nil { t.Fatalf("task not found: %v", err) } - if tk.Agent.ProjectDir != "" { - t.Errorf("expected empty project dir, got %q", tk.Agent.ProjectDir) + if tk.RepositoryURL == "" { + t.Error("expected non-empty repository_url from GitHub webhook payload") } } diff --git a/internal/executor/container.go b/internal/executor/container.go index ba0c03a..2c5b7d3 100644 --- a/internal/executor/container.go +++ b/internal/executor/container.go @@ -48,20 +48,7 @@ func (r *ContainerRunner) Run(ctx context.Context, t *task.Task, e *storage.Exec var err error repoURL := t.RepositoryURL if repoURL == "" { - repoURL = t.Agent.RepositoryURL - } - if repoURL == "" { - // Fallback to project_dir if repository_url is not set (legacy support). - // Prefer the 'local' bare remote so that git push succeeds after execution - // (pushing to a non-bare working copy on a checked-out branch is rejected by git). - if t.Agent.ProjectDir != "" { - repoURL = t.Agent.ProjectDir - if out, err2 := exec.Command("git", "-C", t.Agent.ProjectDir, "remote", "get-url", "local").Output(); err2 == nil { - repoURL = strings.TrimSpace(string(out)) - } - } else { - return fmt.Errorf("task %s has no repository_url or project_dir", t.ID) - } + return fmt.Errorf("task %s has no repository_url", t.ID) } image := t.Agent.ContainerImage @@ -362,25 +349,3 @@ func (r *ContainerRunner) buildInnerCmd(t *task.Task, e *storage.Execution, isRe return []string{"sh", "-c", claudeCmd.String()} } - -func (r *ContainerRunner) fallbackGitInit(repoURL, workspace string) error { - // Ensure directory exists - if err := os.MkdirAll(workspace, 0755); err != nil { - return err - } - // If it's a local directory but not a repo, init it. - cmds := [][]string{ - gitSafe("-C", workspace, "init"), - gitSafe("-C", workspace, "add", "-A"), - gitSafe("-C", workspace, "commit", "--allow-empty", "-m", "chore: initial commit"), - } - // If it was a local path, maybe we should have copied it? - // git clone handle local paths fine if they are repos. - // This fallback is only if it's NOT a repo. - for _, args := range cmds { - if out, err := r.command(context.Background(), "git", args...).CombinedOutput(); err != nil { - return fmt.Errorf("git init failed: %w\n%s", err, out) - } - } - return nil -} diff --git a/internal/storage/db.go b/internal/storage/db.go index 1a0e74f..8f834b2 100644 --- a/internal/storage/db.go +++ b/internal/storage/db.go @@ -109,6 +109,16 @@ func (s *DB) migrate() error { )`, `CREATE INDEX IF NOT EXISTS idx_agent_events_agent ON agent_events(agent)`, `CREATE INDEX IF NOT EXISTS idx_agent_events_timestamp ON agent_events(timestamp)`, + `CREATE TABLE IF NOT EXISTS projects ( + id TEXT PRIMARY KEY, + name TEXT NOT NULL, + remote_url TEXT NOT NULL DEFAULT '', + local_path TEXT NOT NULL DEFAULT '', + type TEXT NOT NULL DEFAULT 'web', + deploy_script TEXT NOT NULL DEFAULT '', + created_at DATETIME NOT NULL, + updated_at DATETIME NOT NULL + )`, } for _, m := range migrations { if _, err := s.db.Exec(m); err != nil { @@ -1056,3 +1066,64 @@ func timeOrNull(t *time.Time) interface{} { } return t.UTC() } + +// CreateProject inserts a new project. +func (s *DB) CreateProject(p *task.Project) error { + now := time.Now().UTC() + _, err := s.db.Exec( + `INSERT INTO projects (id, name, remote_url, local_path, type, deploy_script, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)`, + p.ID, p.Name, p.RemoteURL, p.LocalPath, p.Type, p.DeployScript, now, now, + ) + return err +} + +// GetProject retrieves a project by ID. +func (s *DB) GetProject(id string) (*task.Project, error) { + row := s.db.QueryRow(`SELECT id, name, remote_url, local_path, type, deploy_script FROM projects WHERE id = ?`, id) + p := &task.Project{} + if err := row.Scan(&p.ID, &p.Name, &p.RemoteURL, &p.LocalPath, &p.Type, &p.DeployScript); err != nil { + return nil, err + } + return p, nil +} + +// ListProjects returns all projects. +func (s *DB) ListProjects() ([]*task.Project, error) { + rows, err := s.db.Query(`SELECT id, name, remote_url, local_path, type, deploy_script FROM projects ORDER BY name`) + if err != nil { + return nil, err + } + defer rows.Close() + var projects []*task.Project + for rows.Next() { + p := &task.Project{} + if err := rows.Scan(&p.ID, &p.Name, &p.RemoteURL, &p.LocalPath, &p.Type, &p.DeployScript); err != nil { + return nil, err + } + projects = append(projects, p) + } + return projects, rows.Err() +} + +// UpdateProject updates an existing project. +func (s *DB) UpdateProject(p *task.Project) error { + now := time.Now().UTC() + _, err := s.db.Exec( + `UPDATE projects SET name = ?, remote_url = ?, local_path = ?, type = ?, deploy_script = ?, updated_at = ? WHERE id = ?`, + p.Name, p.RemoteURL, p.LocalPath, p.Type, p.DeployScript, now, p.ID, + ) + return err +} + +// UpsertProject inserts or updates a project by ID (used for seeding). +func (s *DB) UpsertProject(p *task.Project) error { + now := time.Now().UTC() + _, err := s.db.Exec( + `INSERT INTO projects (id, name, remote_url, local_path, type, deploy_script, created_at, updated_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?) + ON CONFLICT(id) DO UPDATE SET name=excluded.name, remote_url=excluded.remote_url, + local_path=excluded.local_path, type=excluded.type, deploy_script=excluded.deploy_script, updated_at=excluded.updated_at`, + p.ID, p.Name, p.RemoteURL, p.LocalPath, p.Type, p.DeployScript, now, now, + ) + return err +} diff --git a/internal/storage/db_test.go b/internal/storage/db_test.go index 5c447af..82c8262 100644 --- a/internal/storage/db_test.go +++ b/internal/storage/db_test.go @@ -41,7 +41,6 @@ func TestCreateTask_AndGetTask(t *testing.T) { Type: "claude", Model: "sonnet", Instructions: "do it", - ProjectDir: "/tmp", MaxBudgetUSD: 2.5, }, Priority: task.PriorityHigh, @@ -1154,3 +1153,68 @@ func TestExecution_StoreAndRetrieveChangestats(t *testing.T) { } } +func TestCreateProject(t *testing.T) { + db := testDB(t) + defer db.Close() + + p := &task.Project{ + ID: "proj-1", + Name: "claudomator", + RemoteURL: "/bare/claudomator.git", + LocalPath: "/workspace/claudomator", + Type: "web", + } + if err := db.CreateProject(p); err != nil { + t.Fatalf("CreateProject: %v", err) + } + got, err := db.GetProject("proj-1") + if err != nil { + t.Fatalf("GetProject: %v", err) + } + if got.Name != "claudomator" { + t.Errorf("Name: want claudomator, got %q", got.Name) + } + if got.LocalPath != "/workspace/claudomator" { + t.Errorf("LocalPath: want /workspace/claudomator, got %q", got.LocalPath) + } +} + +func TestListProjects(t *testing.T) { + db := testDB(t) + defer db.Close() + + for _, p := range []*task.Project{ + {ID: "p1", Name: "alpha", Type: "web"}, + {ID: "p2", Name: "beta", Type: "android"}, + } { + if err := db.CreateProject(p); err != nil { + t.Fatalf("CreateProject: %v", err) + } + } + list, err := db.ListProjects() + if err != nil { + t.Fatalf("ListProjects: %v", err) + } + if len(list) != 2 { + t.Errorf("want 2 projects, got %d", len(list)) + } +} + +func TestUpdateProject(t *testing.T) { + db := testDB(t) + defer db.Close() + + p := &task.Project{ID: "p1", Name: "original", Type: "web"} + if err := db.CreateProject(p); err != nil { + t.Fatalf("CreateProject: %v", err) + } + p.Name = "updated" + if err := db.UpdateProject(p); err != nil { + t.Fatalf("UpdateProject: %v", err) + } + got, _ := db.GetProject("p1") + if got.Name != "updated" { + t.Errorf("Name after update: want updated, got %q", got.Name) + } +} + diff --git a/internal/storage/seed.go b/internal/storage/seed.go new file mode 100644 index 0000000..d1ded8a --- /dev/null +++ b/internal/storage/seed.go @@ -0,0 +1,46 @@ +package storage + +import ( + "os/exec" + "strings" + + "github.com/thepeterstone/claudomator/internal/task" +) + +// SeedProjects upserts the default project registry on startup. +func (s *DB) SeedProjects() error { + projects := []*task.Project{ + { + ID: "claudomator", + Name: "claudomator", + LocalPath: "/workspace/claudomator", + RemoteURL: localBareRemote("/workspace/claudomator"), + Type: "web", + }, + { + ID: "nav", + Name: "nav", + LocalPath: "/workspace/nav", + RemoteURL: localBareRemote("/workspace/nav"), + Type: "android", + }, + } + for _, p := range projects { + if err := s.UpsertProject(p); err != nil { + return err + } + } + return nil +} + +// localBareRemote returns the URL of the "local" git remote for dir, +// falling back to dir itself if the remote is not configured. +func localBareRemote(dir string) string { + out, err := exec.Command("git", "-C", dir, "remote", "get-url", "local").Output() + if err == nil { + if url := strings.TrimSpace(string(out)); url != "" { + return url + } + } + return dir +} diff --git a/internal/task/project.go b/internal/task/project.go new file mode 100644 index 0000000..bd8a4fb --- /dev/null +++ b/internal/task/project.go @@ -0,0 +1,11 @@ +package task + +// Project represents a registered codebase that agents can operate on. +type Project struct { + ID string `json:"id"` + Name string `json:"name"` + RemoteURL string `json:"remote_url"` + LocalPath string `json:"local_path"` + Type string `json:"type"` // "web" | "android" + DeployScript string `json:"deploy_script"` // optional path or command +} diff --git a/internal/task/task.go b/internal/task/task.go index 465de8b..28d65a5 100644 --- a/internal/task/task.go +++ b/internal/task/task.go @@ -32,16 +32,13 @@ type AgentConfig struct { Model string `yaml:"model" json:"model"` ContextFiles []string `yaml:"context_files" json:"context_files"` Instructions string `yaml:"instructions" json:"instructions"` - RepositoryURL string `yaml:"repository_url" json:"repository_url"` ContainerImage string `yaml:"container_image" json:"container_image"` - ProjectDir string `yaml:"project_dir" json:"project_dir"` // Deprecated: use Task.RepositoryURL MaxBudgetUSD float64 `yaml:"max_budget_usd" json:"max_budget_usd"` PermissionMode string `yaml:"permission_mode" json:"permission_mode"` AllowedTools []string `yaml:"allowed_tools" json:"allowed_tools"` DisallowedTools []string `yaml:"disallowed_tools" json:"disallowed_tools"` SystemPromptAppend string `yaml:"system_prompt_append" json:"system_prompt_append"` AdditionalArgs []string `yaml:"additional_args" json:"additional_args"` - SkipPlanning bool `yaml:"skip_planning" json:"skip_planning"` } diff --git a/internal/task/validator_test.go b/internal/task/validator_test.go index 657d93f..c0ab986 100644 --- a/internal/task/validator_test.go +++ b/internal/task/validator_test.go @@ -12,7 +12,6 @@ func validTask() *Task { Agent: AgentConfig{ Type: "claude", Instructions: "do something", - ProjectDir: "/tmp", }, Priority: PriorityNormal, Retry: RetryConfig{MaxAttempts: 1, Backoff: "exponential"}, diff --git a/scripts/check-token b/scripts/check-token new file mode 100644 index 0000000..40a3116 --- /dev/null +++ b/scripts/check-token @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# check-token: Verify Claude OAuth token is valid against the Anthropic API. +# Usage: check-token [--refresh] [--retry-task <id-prefix>] +# --refresh re-authenticate via claude CLI if token is bad +# --retry-task <id> after a successful token check/refresh, retry that task +# +# Exit codes: 0=valid, 1=expired/invalid, 2=credentials file missing + +set -euo pipefail + +CREDS="/root/.claude/.credentials.json" +REFRESH=0 +RETRY_TASK="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --refresh) REFRESH=1; shift ;; + --retry-task) RETRY_TASK="$2"; shift 2 ;; + *) echo "Unknown arg: $1" >&2; exit 2 ;; + esac +done + +if [[ ! -f "$CREDS" ]]; then + echo "ERROR: credentials file not found: $CREDS" >&2 + exit 2 +fi + +ACCESS_TOKEN=$(python3 -c " +import json, sys +d = json.load(open('$CREDS')) +tok = d.get('claudeAiOauth', {}).get('accessToken', '') +if not tok: + print('MISSING', file=sys.stderr) + sys.exit(1) +print(tok) +") + +# Test token against the API with a minimal request +HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \ + -X POST https://api.anthropic.com/v1/messages \ + -H "anthropic-version: 2023-06-01" \ + -H "anthropic-beta: oauth-2025-04-20" \ + -H "Authorization: Bearer $ACCESS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"model":"claude-haiku-4-5-20251001","max_tokens":1,"messages":[{"role":"user","content":"hi"}]}') + +if [[ "$HTTP_STATUS" == "200" ]]; then + echo "OK: token is valid (HTTP $HTTP_STATUS)" + if [[ -n "$RETRY_TASK" ]]; then + /workspace/claudomator/scripts/ct-task "$RETRY_TASK" retry + fi + exit 0 +elif [[ "$HTTP_STATUS" == "401" ]]; then + echo "EXPIRED: token rejected by API (HTTP 401)" + if [[ "$REFRESH" == "1" ]]; then + echo "Re-authenticating via claude CLI..." + claude --dangerously-skip-permissions /dev/null 2>&1 || true + # Check if creds were updated + NEW_TOKEN=$(python3 -c "import json; print(json.load(open('$CREDS')).get('claudeAiOauth',{}).get('accessToken',''))") + if [[ "$NEW_TOKEN" != "$ACCESS_TOKEN" ]]; then + echo "New token obtained. Syncing credentials..." + /workspace/claudomator/scripts/sync-credentials + if [[ -n "$RETRY_TASK" ]]; then + /workspace/claudomator/scripts/ct-task "$RETRY_TASK" retry + fi + exit 0 + else + echo "Token unchanged — manual re-auth required: run 'claude' in a terminal" >&2 + exit 1 + fi + else + echo "Run: check-token --refresh or re-authenticate via 'claude'" >&2 + exit 1 + fi +else + echo "WARN: unexpected HTTP $HTTP_STATUS from API (token may still be valid)" + exit 1 +fi diff --git a/scripts/fix-permissions b/scripts/fix-permissions new file mode 100644 index 0000000..408a23e --- /dev/null +++ b/scripts/fix-permissions @@ -0,0 +1,43 @@ +#!/bin/bash +# claudomator-fix-perms — Fix ownership and permissions for Claudomator components +set -euo pipefail + +SITE_DIR="/site/doot.terst.org" +GIT_REPOS_DIR="/site/git.terst.org/repos" +WORKSPACE_DIR="/workspace" + +echo "==> Fixing site ownership (www-data:www-data)..." +chown -R www-data:www-data "${SITE_DIR}" + +echo "==> Ensuring binaries are executable..." +if [ -d "${SITE_DIR}/bin" ]; then + find "${SITE_DIR}/bin" -type f -exec chmod +x {} + +fi +if [ -f "/usr/local/bin/claudomator" ]; then + chmod +x /usr/local/bin/claudomator +fi + +echo "==> Ensuring scripts are executable..." +if [ -d "${SITE_DIR}/scripts" ]; then + find "${SITE_DIR}/scripts" -type f -exec chmod +x {} + +fi +if [ -d "${WORKSPACE_DIR}/claudomator/scripts" ]; then + find "${WORKSPACE_DIR}/claudomator/scripts" -type f -exec chmod +x {} + +fi + +echo "==> Fixing git bare repo permissions..." +# Specifically fix object permissions that might be corrupted by root runs +if [ -d "${GIT_REPOS_DIR}" ]; then + chown -R www-data:www-data "${GIT_REPOS_DIR}" + find "${GIT_REPOS_DIR}" -type d -exec chmod 775 {} + + find "${GIT_REPOS_DIR}" -type f -exec chmod 664 {} + +fi + +echo "==> Fixing database permissions..." +if [ -f "${SITE_DIR}/data/claudomator.db" ]; then + chmod 664 "${SITE_DIR}/data/claudomator.db" + # Ensure the data directory is writable for WAL mode + chmod 775 "${SITE_DIR}/data" +fi + +echo "==> Done!" diff --git a/scripts/sync-credentials b/scripts/sync-credentials new file mode 100644 index 0000000..78e5311 --- /dev/null +++ b/scripts/sync-credentials @@ -0,0 +1,40 @@ +#!/bin/bash +# sync-credentials — copies Claude and Gemini credentials to workspace + +set -euo pipefail + +# This script is intended to be run by cron every 10 minutes. +# It copies Claude and Gemini credentials from root home to workspace for claudomator. + +# Source paths +SOURCE_CLAUDE="/root/.claude/.credentials.json" +SOURCE_GEMINI_OAUTH="/root/.gemini/oauth_creds.json" +SOURCE_GEMINI_ACCOUNTS="/root/.gemini/google_accounts.json" + +# Destination paths +DEST_CLAUDE="/workspace/claudomator/credentials/claude/.credentials.json" +DEST_GEMINI_OAUTH="/workspace/claudomator/credentials/gemini/oauth_creds.json" +DEST_GEMINI_ACCOUNTS="/workspace/claudomator/credentials/gemini/google_accounts.json" + +# Sync Claude +if [[ -f "$SOURCE_CLAUDE" ]]; then + mkdir -p "$(dirname "$DEST_CLAUDE")" + cp "$SOURCE_CLAUDE" "$DEST_CLAUDE" + chmod 600 "$DEST_CLAUDE" + echo "Synced Claude credentials." +fi + +# Sync Gemini +if [[ -f "$SOURCE_GEMINI_OAUTH" ]]; then + mkdir -p "$(dirname "$DEST_GEMINI_OAUTH")" + cp "$SOURCE_GEMINI_OAUTH" "$DEST_GEMINI_OAUTH" + chmod 600 "$DEST_GEMINI_OAUTH" + echo "Synced Gemini OAuth credentials." +fi + +if [[ -f "$SOURCE_GEMINI_ACCOUNTS" ]]; then + mkdir -p "$(dirname "$DEST_GEMINI_ACCOUNTS")" + cp "$SOURCE_GEMINI_ACCOUNTS" "$DEST_GEMINI_ACCOUNTS" + chmod 600 "$DEST_GEMINI_ACCOUNTS" + echo "Synced Gemini Google accounts." +fi |
