diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/plans/local-oss-runner.md | 57 |
1 files changed, 57 insertions, 0 deletions
diff --git a/docs/plans/local-oss-runner.md b/docs/plans/local-oss-runner.md index 108495b..c065483 100644 --- a/docs/plans/local-oss-runner.md +++ b/docs/plans/local-oss-runner.md @@ -247,3 +247,60 @@ Second-cheapest, second-highest-volume LLM call after classification (one per ta - `prefer_local_for_elaborate=false` short-circuits to Claude path (preserves current behavior when user opts out) - Local-failure fallback to Claude verified by test - Branch pushed + +--- + +# Phase 3 — Focused Plan (CI Failure Triage) + +## Scope adjustment from the original sketch + +The original Phase 3 sketch was "summarize fetched workflow logs". Fetching GitHub workflow logs requires authenticated GitHub API access (PAT or app token), which is out of scope and would balloon this phase into a GitHub-integration epic. Narrow Phase 3 to **project-context-based triage** — use signals we already have without new dependencies. + +What we have at webhook time: `repository.full_name`, `branch`, `SHA`, `check_name`, `html_url`, plus (when matched) a project directory we can read locally. + +What the LLM can do with that: produce a tighter, project-aware investigation prompt that names the recent commits, points at suspect files, and gives the agent better starting hypotheses than the current generic 4-step template. + +## What ships + +- New helper `enrichCIInstructions(ctx, *llm.Client, ciContext, projectDir, fallback string) string` +- `createCIFailureTask` calls it when `s.llm != nil`; on any error, returns the existing hardcoded template (truly additive — webhook tests for the no-LLM path stay passing unchanged) +- Helper uses: recent git log (last 5 commits from project_dir if it's a git repo), CLAUDE.md content if present, plus all webhook metadata +- One configuration knob: reuse `LocalModel.UseForElaborate()` semantics? No — separate flag. Add `LocalModel.PreferForCITriage *bool` defaulting true when endpoint set, opt-out symmetrical with `PreferForElaborate`. + +## Explicit non-goals + +- No GitHub API integration (no log fetching, no auth) +- No changes to webhook routing, signature validation, project matching, or task scheduling +- No changes to the task schema (instructions stays a string) +- No streaming — one-shot LLM call, sub-2s target + +## Task list + +1. Add `LocalModel.PreferForCITriage *bool` and `UseForCITriage()` helper, mirroring elaborate +2. Add `enrichCIInstructions` in `internal/api/webhook.go` (or `webhook_llm.go` if it grows) +3. Read recent git log from project_dir via `git log --oneline -n 5` (best-effort, swallow errors) +4. Read CLAUDE.md from project_dir (best-effort) +5. Build a focused prompt: "CI just failed on this project. Here's metadata + recent commits + project context. Produce a 6-12 line investigation plan that names suspect files/commits when you can, otherwise gives concrete starting steps." Plain text out, not JSON. +6. Update `createCIFailureTask` to call enrichment when `s.llm != nil && cfg.LocalModel.UseForCITriage()`. Note: the server doesn't currently see the cfg directly — pass the gate as a setter `SetCITriageEnabled(bool)` from serve.go, OR (simpler) just gate on `s.llm != nil` and let users opt out by not calling `SetLLM`. Going with the simpler option since it matches the elaborate split: same `s.llm` for both, server doesn't track per-feature gates. +7. Wiring in `serve.go`: when `cfg.LocalModel.Endpoint != ""`, `SetLLM(localClient)`. (Already done in Phase 2.) Per-feature opt-out via the `PreferFor*` config flags is read at wire time and could conditionally not call SetLLM, but that gives elaborate/CI an all-or-nothing toggle which is wrong. Better: introduce a separate setter `SetLLMForCITriage` so each feature can be controlled independently. + + Actually, simplest and cleanest: keep one `SetLLM` setter, and gate each call site (`elaborateWithLocal`, `enrichCIInstructions`) by reading a per-feature config flag passed via separate setters. That's getting fiddly. Step back. + + **Final decision:** the per-feature gate doesn't pull its weight in Phase 3. Ship it as: `s.llm != nil` enables both elaborate and CI triage. Users who want elaborate-yes/CI-triage-no can revisit later. The deferred per-feature toggles get added in the post-epic cleanup along with token telemetry — there's no real demand for the asymmetric case yet. + + Revised: drop `PreferForCITriage` entirely; ship a simpler thing. +8. Tests: + - `enrichCIInstructions` with stub LLM returns the LLM body + - `enrichCIInstructions` with failing LLM returns `fallback` unchanged + - `enrichCIInstructions` includes recent git log when project_dir is a real git repo (use `t.TempDir()` + `git init` + a commit) + - Webhook handler test: LLM configured → instructions reflect LLM output + - Webhook handler test: LLM not configured → instructions match the existing template byte-for-byte (regression guard) +9. `go build ./... && go test -race ./...` +10. Commit as Phase 3 on the same branch +11. Push + +## Stop conditions + +- All new tests green under `-race` +- Existing webhook tests pass byte-for-byte when LLM not configured +- Build clean; pushed |