diff options
| author | Claude <noreply@anthropic.com> | 2026-04-28 17:10:27 +0000 |
|---|---|---|
| committer | Claude <noreply@anthropic.com> | 2026-04-28 17:10:27 +0000 |
| commit | ae833b2765c7c8086bf8e1ea8e8ec8ee9b73e656 (patch) | |
| tree | b2cda4dc982d6c04eb22033e19645091af42224b /internal/config/config.go | |
| parent | 0865afc43be562dbe14528e4299b9e213b54cc93 (diff) | |
feat(api): route elaboration through local LLM when configured
Phase 2 of "local OSS models as agents" plan. Adds a third elaboration
path that calls the local OpenAI-compatible LLM via the internal/llm
client, and reorders dispatch so the cheap path is tried first:
local → claude → gemini, with each next attempt only on hard failure
of the prior.
Wiring is opt-out, not opt-in: when [local_model].endpoint is set,
elaboration prefers local by default. Users with a slow or low-quality
local model can disable just elaboration via:
[local_model]
endpoint = "..."
prefer_for_elaborate = false
without giving up the runner or the classifier path.
Implementation:
- Server gains an optional *llm.Client field via SetLLM (matches the
existing SetNotifier/SetWorkspaceRoot setter pattern, no NewServer
signature break).
- elaborateWithLocal() reuses buildElaboratePrompt verbatim and asks
for response_format=json_object so we skip markdown-fence cleanup.
- handleElaborateTask reorders try chain; existing Claude-first
behavior is preserved exactly when SetLLM is not called.
- LocalModel.UseForElaborate() encapsulates the default-true gating
with a *bool so explicit-false survives TOML parse.
Tests:
- elaborateWithLocal: parses valid response, errors on nil client,
errors on bad JSON.
- handler: local preferred when wired; falls back to claude when
local fails; unchanged behavior when no LLM is configured.
- config: UseForElaborate gating across empty/default/explicit-true/
explicit-false cases.
Pre-existing test failures noted in docs/plans/local-oss-runner.md
(post-epic cleanup): TestGeminiLogs_ParsedCorrectly returns 404 for
gemini execution log fetch — predates this change.
Plan: docs/plans/local-oss-runner.md.
https://claude.ai/code/session_017Edeq947TpSm1vQTxMhi1J
Diffstat (limited to 'internal/config/config.go')
| -rw-r--r-- | internal/config/config.go | 33 |
1 files changed, 25 insertions, 8 deletions
diff --git a/internal/config/config.go b/internal/config/config.go index 7f87391..5801239 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -16,15 +16,32 @@ type Project struct { } // LocalModel configures an OpenAI-compatible local LLM endpoint used for -// internal helpers (classifier, future elaboration/summarization) and as the -// backend for the "local" runner. If Endpoint is empty, the LocalRunner is -// not registered and the classifier falls back to the Gemini CLI. +// internal helpers (classifier, elaboration, future summarization) and as +// the backend for the "local" runner. If Endpoint is empty, the LocalRunner +// is not registered and the classifier falls back to the Gemini CLI. +// +// PreferForElaborate gates whether the API server's elaboration handler +// uses this client. It defaults to true when Endpoint is set; users with a +// slow or low-quality local model can disable it. type LocalModel struct { - Endpoint string `toml:"endpoint"` // e.g. "http://localhost:11434/v1" - Model string `toml:"model"` // e.g. "llama3.1:8b" - TimeoutSeconds int `toml:"timeout_seconds"` // default 60 - DefaultTemperature float64 `toml:"default_temperature"` // default 0.2 - APIKey string `toml:"api_key"` // optional bearer token + Endpoint string `toml:"endpoint"` // e.g. "http://localhost:11434/v1" + Model string `toml:"model"` // e.g. "llama3.1:8b" + TimeoutSeconds int `toml:"timeout_seconds"` // default 60 + DefaultTemperature float64 `toml:"default_temperature"` // default 0.2 + APIKey string `toml:"api_key"` // optional bearer token + PreferForElaborate *bool `toml:"prefer_for_elaborate"` // pointer so default-true survives parse +} + +// UseForElaborate returns true when elaboration should try this local model +// before falling back to Claude/Gemini. Default is true when Endpoint is set. +func (m LocalModel) UseForElaborate() bool { + if m.Endpoint == "" { + return false + } + if m.PreferForElaborate == nil { + return true + } + return *m.PreferForElaborate } type Config struct { |
