feat(api): enrich CI failure task instructions via local LLM

Phase 3 of "local OSS models as agents" plan. When the webhook handler
creates a task for a failed CI run AND a local LLM is configured on
the server, the hardcoded 4-step investigation template is replaced
with a project-aware investigation plan generated by the LLM.

Scope adjustment from the original sketch: the original plan said
"summarize fetched workflow logs", but fetching logs requires GitHub
API auth that isn't wired. Narrowed to project-context triage —
recent git log + CLAUDE.md content + webhook metadata, fed to the
LLM with a system prompt asking for 6-12 lines of concrete next
steps. Deferred GitHub log fetching to post-epic cleanup.

Implementation:
- New internal/api/webhook_llm.go holds enrichCIInstructions and its
  helpers (readRecentCommits via `git log`, readProjectDoc).
- enrichCIInstructions is truly additive: any failure mode (no client,
  HTTP error, empty body, 10s timeout) returns the original fallback
  template unchanged. Existing webhook tests pass byte-for-byte.
- Always preserves a metadata header (repo/branch/SHA/check/URL)
  ahead of the LLM body so investigators don't lose context if the
  LLM is terse.
- Reuses s.llm (set via Server.SetLLM in Phase 2) — no new config
  knob, no per-feature gating. Asymmetric opt-out (yes-elaborate,
  no-CI-triage) deferred until there's actual demand.

Tests:
- enrichCIInstructions: nil client, LLM 500, empty body all return
  fallback unchanged.
- enrichCIInstructions: success path produces enriched body with
  metadata header preserved; user prompt contains repo/branch/SHA.
- enrichCIInstructions: real git repo (init + 2 commits) → recent
  commits appear in user prompt.
- Webhook handler regression guard: no-LLM path produces the exact
  legacy template substrings.
- Webhook handler with LLM stubbed: task instructions contain LLM
  body + metadata header.

Plan: docs/plans/local-oss-runner.md.

https://claude.ai/code/session_017Edeq947TpSm1vQTxMhi1J

4 files changed, 426 insertions, 1 deletions

diff --git a/docs/plans/local-oss-runner.md b/docs/plans/local-oss-runner.md
index 108495b..c065483 100644
--- a/docs/plans/local-oss-runner.md
+++ b/docs/plans/local-oss-runner.md
@@ -247,3 +247,60 @@ Second-cheapest, second-highest-volume LLM call after classification (one per ta
 - `prefer_local_for_elaborate=false` short-circuits to Claude path (preserves current behavior when user opts out)
 - Local-failure fallback to Claude verified by test
 - Branch pushed
+
+---
+
+# Phase 3 — Focused Plan (CI Failure Triage)
+
+## Scope adjustment from the original sketch
+
+The original Phase 3 sketch was "summarize fetched workflow logs". Fetching GitHub workflow logs requires authenticated GitHub API access (PAT or app token), which is out of scope and would balloon this phase into a GitHub-integration epic. Narrow Phase 3 to **project-context-based triage** — use signals we already have without new dependencies.
+
+What we have at webhook time: `repository.full_name`, `branch`, `SHA`, `check_name`, `html_url`, plus (when matched) a project directory we can read locally.
+
+What the LLM can do with that: produce a tighter, project-aware investigation prompt that names the recent commits, points at suspect files, and gives the agent better starting hypotheses than the current generic 4-step template.
+
+## What ships
+
+- New helper `enrichCIInstructions(ctx, *llm.Client, ciContext, projectDir, fallback string) string`
+- `createCIFailureTask` calls it when `s.llm != nil`; on any error, returns the existing hardcoded template (truly additive — webhook tests for the no-LLM path stay passing unchanged)
+- Helper uses: recent git log (last 5 commits from project_dir if it's a git repo), CLAUDE.md content if present, plus all webhook metadata
+- One configuration knob: reuse `LocalModel.UseForElaborate()` semantics? No — separate flag. Add `LocalModel.PreferForCITriage *bool` defaulting true when endpoint set, opt-out symmetrical with `PreferForElaborate`.
+
+## Explicit non-goals
+
+- No GitHub API integration (no log fetching, no auth)
+- No changes to webhook routing, signature validation, project matching, or task scheduling
+- No changes to the task schema (instructions stays a string)
+- No streaming — one-shot LLM call, sub-2s target
+
+## Task list
+
+1. Add `LocalModel.PreferForCITriage *bool` and `UseForCITriage()` helper, mirroring elaborate
+2. Add `enrichCIInstructions` in `internal/api/webhook.go` (or `webhook_llm.go` if it grows)
+3. Read recent git log from project_dir via `git log --oneline -n 5` (best-effort, swallow errors)
+4. Read CLAUDE.md from project_dir (best-effort)
+5. Build a focused prompt: "CI just failed on this project. Here's metadata + recent commits + project context. Produce a 6-12 line investigation plan that names suspect files/commits when you can, otherwise gives concrete starting steps." Plain text out, not JSON.
+6. Update `createCIFailureTask` to call enrichment when `s.llm != nil && cfg.LocalModel.UseForCITriage()`. Note: the server doesn't currently see the cfg directly — pass the gate as a setter `SetCITriageEnabled(bool)` from serve.go, OR (simpler) just gate on `s.llm != nil` and let users opt out by not calling `SetLLM`. Going with the simpler option since it matches the elaborate split: same `s.llm` for both, server doesn't track per-feature gates.
+7. Wiring in `serve.go`: when `cfg.LocalModel.Endpoint != ""`, `SetLLM(localClient)`. (Already done in Phase 2.) Per-feature opt-out via the `PreferFor*` config flags is read at wire time and could conditionally not call SetLLM, but that gives elaborate/CI an all-or-nothing toggle which is wrong. Better: introduce a separate setter `SetLLMForCITriage` so each feature can be controlled independently.
+
+   Actually, simplest and cleanest: keep one `SetLLM` setter, and gate each call site (`elaborateWithLocal`, `enrichCIInstructions`) by reading a per-feature config flag passed via separate setters. That's getting fiddly. Step back.
+
+   **Final decision:** the per-feature gate doesn't pull its weight in Phase 3. Ship it as: `s.llm != nil` enables both elaborate and CI triage. Users who want elaborate-yes/CI-triage-no can revisit later. The deferred per-feature toggles get added in the post-epic cleanup along with token telemetry — there's no real demand for the asymmetric case yet.
+
+   Revised: drop `PreferForCITriage` entirely; ship a simpler thing.
+8. Tests:
+   - `enrichCIInstructions` with stub LLM returns the LLM body
+   - `enrichCIInstructions` with failing LLM returns `fallback` unchanged
+   - `enrichCIInstructions` includes recent git log when project_dir is a real git repo (use `t.TempDir()` + `git init` + a commit)
+   - Webhook handler test: LLM configured → instructions reflect LLM output
+   - Webhook handler test: LLM not configured → instructions match the existing template byte-for-byte (regression guard)
+9. `go build ./... && go test -race ./...`
+10. Commit as Phase 3 on the same branch
+11. Push
+
+## Stop conditions
+
+- All new tests green under `-race`
+- Existing webhook tests pass byte-for-byte when LLM not configured
+- Build clean; pushed
diff --git a/internal/api/webhook.go b/internal/api/webhook.go
index 8bf1676..9437f7d 100644
--- a/internal/api/webhook.go
+++ b/internal/api/webhook.go
@@ -1,6 +1,7 @@
 package api
 
 import (
+	"context"
 	"crypto/hmac"
 	"crypto/sha256"
 	"encoding/hex"
@@ -154,7 +155,7 @@ func (s *Server) handleWorkflowRunEvent(w http.ResponseWriter, body []byte) {
 func (s *Server) createCIFailureTask(w http.ResponseWriter, repoName, fullName, branch, sha, checkName, htmlURL string) {
 	project := matchProject(s.projects, repoName)
 
-	instructions := fmt.Sprintf(
+	fallback := fmt.Sprintf(
 		"A CI failure has been detected and requires investigation.\n\n"+
 			"Repository: %s\n"+
 			"Branch: %s\n"+
@@ -169,6 +170,18 @@ func (s *Server) createCIFailureTask(w http.ResponseWriter, repoName, fullName,
 		fullName, branch, sha, checkName, htmlURL,
 	)
 
+	tctx := ciTriageContext{
+		Repo:      fullName,
+		Branch:    branch,
+		SHA:       sha,
+		CheckName: checkName,
+		URL:       htmlURL,
+	}
+	if project != nil {
+		tctx.ProjectDir = project.Dir
+	}
+	instructions := enrichCIInstructions(context.Background(), s.llm, tctx, fallback)
+
 	now := time.Now().UTC()
 	t := &task.Task{
 		ID:   uuid.New().String(),
diff --git a/internal/api/webhook_llm.go b/internal/api/webhook_llm.go
new file mode 100644
index 0000000..1cbca17
--- /dev/null
+++ b/internal/api/webhook_llm.go
@@ -0,0 +1,127 @@
+package api
+
+import (
+	"context"
+	"fmt"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"github.com/thepeterstone/claudomator/internal/llm"
+)
+
+// ciTriagePromptTimeout caps the LLM enrichment call so a slow local model
+// can't stall webhook handling. On timeout the original template is used.
+const ciTriagePromptTimeout = 10 * time.Second
+
+// ciTriageContext holds everything we know at webhook time, plus best-effort
+// project-side signals (recent git log, CLAUDE.md content) when project_dir
+// is available.
+type ciTriageContext struct {
+	Repo         string
+	Branch       string
+	SHA          string
+	CheckName    string
+	URL          string
+	ProjectDir   string
+	RecentCommits string // multi-line, may be ""
+	ProjectDoc    string // first ~4 KB of CLAUDE.md, may be ""
+}
+
+// enrichCIInstructions asks the local LLM to produce a tighter, project-aware
+// investigation plan than the hardcoded template. On any error (no client,
+// timeout, parse failure) it returns fallback unchanged so the webhook flow
+// is never worse off for trying.
+func enrichCIInstructions(parent context.Context, c *llm.Client, ctx ciTriageContext, fallback string) string {
+	if c == nil {
+		return fallback
+	}
+
+	// Pull project-side signals best-effort. Errors are silently swallowed —
+	// the LLM still gets the metadata it does have.
+	if ctx.ProjectDir != "" {
+		ctx.RecentCommits = readRecentCommits(ctx.ProjectDir, 5)
+		ctx.ProjectDoc = readProjectDoc(ctx.ProjectDir)
+	}
+
+	cctx, cancel := context.WithTimeout(parent, ciTriagePromptTimeout)
+	defer cancel()
+
+	prompt := buildCITriagePrompt(ctx)
+	resp, err := c.Chat(cctx, llm.ChatRequest{
+		Messages: []llm.Message{
+			{Role: "system", Content: "You produce concise, actionable CI failure investigation plans. Respond with plain text only — no markdown fences, no JSON, no preamble."},
+			{Role: "user", Content: prompt},
+		},
+	})
+	if err != nil {
+		return fallback
+	}
+	body := strings.TrimSpace(resp.Content)
+	if body == "" {
+		return fallback
+	}
+	// Always preserve the metadata header from the fallback so investigators
+	// can see repo/branch/SHA/URL even if the LLM body is terse.
+	return ciInstructionsHeader(ctx) + "\n\n" + body
+}
+
+func buildCITriagePrompt(ctx ciTriageContext) string {
+	var sb strings.Builder
+	fmt.Fprintf(&sb, "CI just failed.\n\nRepository: %s\nBranch: %s\nCommit SHA: %s\nCheck/Workflow: %s\nRun URL: %s\n",
+		ctx.Repo, ctx.Branch, ctx.SHA, ctx.CheckName, ctx.URL)
+	if ctx.RecentCommits != "" {
+		fmt.Fprintf(&sb, "\nRecent commits on this branch (newest first):\n%s\n", ctx.RecentCommits)
+	}
+	if ctx.ProjectDoc != "" {
+		fmt.Fprintf(&sb, "\nProject context (CLAUDE.md, truncated):\n%s\n", ctx.ProjectDoc)
+	}
+	sb.WriteString("\nProduce 6–12 lines of investigation steps. Name suspect commits or files when you can; otherwise give concrete starting actions (which logs to read, which tests to re-run locally). End with an explicit 'Acceptance Criteria' section listing what 'fixed' looks like.")
+	return sb.String()
+}
+
+func ciInstructionsHeader(ctx ciTriageContext) string {
+	return fmt.Sprintf(
+		"A CI failure has been detected and requires investigation.\n\n"+
+			"Repository: %s\n"+
+			"Branch: %s\n"+
+			"Commit SHA: %s\n"+
+			"Check/Workflow: %s\n"+
+			"Run URL: %s",
+		ctx.Repo, ctx.Branch, ctx.SHA, ctx.CheckName, ctx.URL,
+	)
+}
+
+// readRecentCommits returns the last n commits as a `git log --oneline`-style
+// string, or "" on any error.
+func readRecentCommits(projectDir string, n int) string {
+	if projectDir == "" {
+		return ""
+	}
+	cctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
+	defer cancel()
+	cmd := exec.CommandContext(cctx, "git", "-C", projectDir, "log", "--oneline", fmt.Sprintf("-n%d", n))
+	out, err := cmd.Output()
+	if err != nil {
+		return ""
+	}
+	return strings.TrimSpace(string(out))
+}
+
+// readProjectDoc returns CLAUDE.md content (capped at 4KB) or "".
+func readProjectDoc(projectDir string) string {
+	if projectDir == "" {
+		return ""
+	}
+	data, err := os.ReadFile(filepath.Join(projectDir, "CLAUDE.md"))
+	if err != nil {
+		return ""
+	}
+	const cap = 4096
+	if len(data) > cap {
+		data = data[:cap]
+	}
+	return strings.TrimSpace(string(data))
+}
diff --git a/internal/api/webhook_llm_test.go b/internal/api/webhook_llm_test.go
new file mode 100644
index 0000000..f2381a1
--- /dev/null
+++ b/internal/api/webhook_llm_test.go
@@ -0,0 +1,228 @@
+package api
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/thepeterstone/claudomator/internal/config"
+	"github.com/thepeterstone/claudomator/internal/llm"
+)
+
+// initGitRepo creates a fresh git repo with two commits and returns its path.
+// Used to verify enrichCIInstructions picks up recent commits.
+func initGitRepo(t *testing.T) string {
+	t.Helper()
+	dir := t.TempDir()
+	run := func(args ...string) {
+		cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
+		cmd.Env = append(os.Environ(),
+			"GIT_AUTHOR_NAME=test", "GIT_AUTHOR_EMAIL=test@example.com",
+			"GIT_COMMITTER_NAME=test", "GIT_COMMITTER_EMAIL=test@example.com",
+			// Disable signing in case the host has a global pre-commit signer.
+			"GIT_CONFIG_GLOBAL=/dev/null",
+		)
+		if out, err := cmd.CombinedOutput(); err != nil {
+			t.Fatalf("git %v: %v\n%s", args, err, out)
+		}
+	}
+	run("init", "-q")
+	run("config", "commit.gpgsign", "false")
+	run("config", "tag.gpgsign", "false")
+	if err := os.WriteFile(filepath.Join(dir, "README"), []byte("v1\n"), 0644); err != nil {
+		t.Fatal(err)
+	}
+	run("add", "README")
+	run("commit", "-q", "-m", "first commit", "--no-gpg-sign")
+	if err := os.WriteFile(filepath.Join(dir, "README"), []byte("v2\n"), 0644); err != nil {
+		t.Fatal(err)
+	}
+	run("add", "README")
+	run("commit", "-q", "-m", "fix: bump readme", "--no-gpg-sign")
+	return dir
+}
+
+func TestEnrichCIInstructions_NilClient_ReturnsFallback(t *testing.T) {
+	got := enrichCIInstructions(context.Background(), nil, ciTriageContext{}, "FALLBACK")
+	if got != "FALLBACK" {
+		t.Errorf("nil client: want FALLBACK, got %q", got)
+	}
+}
+
+func TestEnrichCIInstructions_LLMFailure_ReturnsFallback(t *testing.T) {
+	// Server that always 500s.
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		http.Error(w, "boom", http.StatusInternalServerError)
+	}))
+	defer srv.Close()
+
+	c := &llm.Client{Endpoint: srv.URL + "/v1", Model: "fake"}
+	got := enrichCIInstructions(context.Background(), c,
+		ciTriageContext{Repo: "x", Branch: "main"}, "FALLBACK")
+	if got != "FALLBACK" {
+		t.Errorf("llm failure: want FALLBACK, got %q", got)
+	}
+}
+
+func TestEnrichCIInstructions_EmptyLLMBody_ReturnsFallback(t *testing.T) {
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/json")
+		fmt.Fprintln(w, `{"model":"x","choices":[{"message":{"content":""},"finish_reason":"stop"}],"usage":{}}`)
+	}))
+	defer srv.Close()
+	c := &llm.Client{Endpoint: srv.URL + "/v1", Model: "fake"}
+	got := enrichCIInstructions(context.Background(), c, ciTriageContext{}, "FALLBACK-2")
+	if got != "FALLBACK-2" {
+		t.Errorf("empty body: want fallback, got %q", got)
+	}
+}
+
+func TestEnrichCIInstructions_LLMSuccess_ReturnsEnriched(t *testing.T) {
+	expected := "1. Look at commit abc123\n2. Re-run build locally\n3. Check unit tests"
+
+	var capturedPrompt string
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		var body struct {
+			Messages []struct {
+				Role    string `json:"role"`
+				Content string `json:"content"`
+			} `json:"messages"`
+		}
+		if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
+			t.Fatal(err)
+		}
+		// Capture the user message so we can assert metadata is in the prompt.
+		for _, m := range body.Messages {
+			if m.Role == "user" {
+				capturedPrompt = m.Content
+			}
+		}
+
+		w.Header().Set("Content-Type", "application/json")
+		fmt.Fprintf(w, `{"model":"x","choices":[{"message":{"content":%q},"finish_reason":"stop"}],"usage":{}}`, expected)
+	}))
+	defer srv.Close()
+
+	c := &llm.Client{Endpoint: srv.URL + "/v1", Model: "fake"}
+	tctx := ciTriageContext{
+		Repo:      "owner/myrepo",
+		Branch:    "main",
+		SHA:       "abc123",
+		CheckName: "CI Build",
+		URL:       "https://github.com/owner/myrepo/runs/1",
+	}
+	got := enrichCIInstructions(context.Background(), c, tctx, "FALLBACK")
+
+	if !strings.Contains(got, expected) {
+		t.Errorf("enriched body missing LLM content; got: %s", got)
+	}
+	if !strings.Contains(got, "Repository: owner/myrepo") {
+		t.Errorf("enriched body missing metadata header; got: %s", got)
+	}
+	for _, want := range []string{"owner/myrepo", "main", "abc123", "CI Build"} {
+		if !strings.Contains(capturedPrompt, want) {
+			t.Errorf("prompt missing %q; got: %s", want, capturedPrompt)
+		}
+	}
+}
+
+func TestEnrichCIInstructions_IncludesRecentCommits(t *testing.T) {
+	repo := initGitRepo(t)
+
+	var capturedPrompt string
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		var body struct {
+			Messages []struct {
+				Role    string `json:"role"`
+				Content string `json:"content"`
+			} `json:"messages"`
+		}
+		json.NewDecoder(r.Body).Decode(&body)
+		for _, m := range body.Messages {
+			if m.Role == "user" {
+				capturedPrompt = m.Content
+			}
+		}
+		w.Header().Set("Content-Type", "application/json")
+		fmt.Fprintln(w, `{"model":"x","choices":[{"message":{"content":"plan"},"finish_reason":"stop"}],"usage":{}}`)
+	}))
+	defer srv.Close()
+
+	c := &llm.Client{Endpoint: srv.URL + "/v1", Model: "fake"}
+	enrichCIInstructions(context.Background(), c,
+		ciTriageContext{Repo: "x", Branch: "y", ProjectDir: repo}, "FALLBACK")
+
+	if !strings.Contains(capturedPrompt, "Recent commits") {
+		t.Errorf("expected prompt to include recent commits section; got:\n%s", capturedPrompt)
+	}
+	if !strings.Contains(capturedPrompt, "fix: bump readme") {
+		t.Errorf("expected most recent commit message in prompt; got:\n%s", capturedPrompt)
+	}
+}
+
+// TestWebhook_NoLLM_InstructionsPreserved is the regression guard: when no
+// LLM is configured, webhook task instructions match the historical template
+// exactly.
+func TestWebhook_NoLLM_InstructionsPreserved(t *testing.T) {
+	srv, store := testServer(t)
+	srv.projects = []config.Project{{Name: "myrepo", Dir: "/workspace/myrepo"}}
+
+	w := webhookPost(t, srv, "check_run", checkRunFailurePayload, "")
+	if w.Code != http.StatusOK {
+		t.Fatalf("status: %d", w.Code)
+	}
+	var resp map[string]string
+	json.NewDecoder(w.Body).Decode(&resp)
+	tk, err := store.GetTask(resp["task_id"])
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, want := range []string{
+		"A CI failure has been detected",
+		"Please investigate the failure by:",
+		"1. Reviewing recent commits on the branch",
+		"4. Fixing the root cause and ensuring the build passes",
+	} {
+		if !strings.Contains(tk.Agent.Instructions, want) {
+			t.Errorf("instructions missing %q (regression: LLM path leaked into no-LLM case)", want)
+		}
+	}
+}
+
+// TestWebhook_WithLLM_InstructionsEnriched verifies the LLM body appears in
+// the created task's instructions when SetLLM is configured.
+func TestWebhook_WithLLM_InstructionsEnriched(t *testing.T) {
+	srv, store := testServer(t)
+	srv.projects = []config.Project{{Name: "myrepo", Dir: "/workspace/myrepo"}}
+
+	llmSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/json")
+		fmt.Fprintln(w, `{"model":"x","choices":[{"message":{"content":"LLM-GENERATED-PLAN"},"finish_reason":"stop"}],"usage":{}}`)
+	}))
+	defer llmSrv.Close()
+	srv.SetLLM(&llm.Client{Endpoint: llmSrv.URL + "/v1", Model: "fake"})
+
+	w := webhookPost(t, srv, "check_run", checkRunFailurePayload, "")
+	if w.Code != http.StatusOK {
+		t.Fatalf("status: %d body: %s", w.Code, w.Body.String())
+	}
+	var resp map[string]string
+	json.NewDecoder(w.Body).Decode(&resp)
+	tk, err := store.GetTask(resp["task_id"])
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !strings.Contains(tk.Agent.Instructions, "LLM-GENERATED-PLAN") {
+		t.Errorf("instructions missing LLM body; got:\n%s", tk.Agent.Instructions)
+	}
+	if !strings.Contains(tk.Agent.Instructions, "Repository: owner/myrepo") {
+		t.Errorf("instructions missing metadata header; got:\n%s", tk.Agent.Instructions)
+	}
+}