From ae833b2765c7c8086bf8e1ea8e8ec8ee9b73e656 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 28 Apr 2026 17:10:27 +0000 Subject: feat(api): route elaboration through local LLM when configured MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2 of "local OSS models as agents" plan. Adds a third elaboration path that calls the local OpenAI-compatible LLM via the internal/llm client, and reorders dispatch so the cheap path is tried first: local → claude → gemini, with each next attempt only on hard failure of the prior. Wiring is opt-out, not opt-in: when [local_model].endpoint is set, elaboration prefers local by default. Users with a slow or low-quality local model can disable just elaboration via: [local_model] endpoint = "..." prefer_for_elaborate = false without giving up the runner or the classifier path. Implementation: - Server gains an optional *llm.Client field via SetLLM (matches the existing SetNotifier/SetWorkspaceRoot setter pattern, no NewServer signature break). - elaborateWithLocal() reuses buildElaboratePrompt verbatim and asks for response_format=json_object so we skip markdown-fence cleanup. - handleElaborateTask reorders try chain; existing Claude-first behavior is preserved exactly when SetLLM is not called. - LocalModel.UseForElaborate() encapsulates the default-true gating with a *bool so explicit-false survives TOML parse. Tests: - elaborateWithLocal: parses valid response, errors on nil client, errors on bad JSON. - handler: local preferred when wired; falls back to claude when local fails; unchanged behavior when no LLM is configured. - config: UseForElaborate gating across empty/default/explicit-true/ explicit-false cases. Pre-existing test failures noted in docs/plans/local-oss-runner.md (post-epic cleanup): TestGeminiLogs_ParsedCorrectly returns 404 for gemini execution log fetch — predates this change. Plan: docs/plans/local-oss-runner.md. https://claude.ai/code/session_017Edeq947TpSm1vQTxMhi1J --- internal/cli/serve.go | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'internal/cli') diff --git a/internal/cli/serve.go b/internal/cli/serve.go index e183bfc..2263d01 100644 --- a/internal/cli/serve.go +++ b/internal/cli/serve.go @@ -99,6 +99,10 @@ func serve(addr string) error { if cfg.WorkspaceRoot != "" { srv.SetWorkspaceRoot(cfg.WorkspaceRoot) } + if cfg.LocalModel.UseForElaborate() { + srv.SetLLM(localClient) + logger.Info("elaboration prefers local llm", "endpoint", cfg.LocalModel.Endpoint) + } srv.SetGitHubWebhookConfig(cfg.WebhookSecret, cfg.Projects) // Register scripts. -- cgit v1.2.3