summaryrefslogtreecommitdiff
path: root/internal/config/config.go
diff options
context:
space:
mode:
authorClaude <noreply@anthropic.com>2026-04-28 17:10:27 +0000
committerClaude <noreply@anthropic.com>2026-04-28 17:10:27 +0000
commitae833b2765c7c8086bf8e1ea8e8ec8ee9b73e656 (patch)
treeb2cda4dc982d6c04eb22033e19645091af42224b /internal/config/config.go
parent0865afc43be562dbe14528e4299b9e213b54cc93 (diff)
feat(api): route elaboration through local LLM when configured
Phase 2 of "local OSS models as agents" plan. Adds a third elaboration path that calls the local OpenAI-compatible LLM via the internal/llm client, and reorders dispatch so the cheap path is tried first: local → claude → gemini, with each next attempt only on hard failure of the prior. Wiring is opt-out, not opt-in: when [local_model].endpoint is set, elaboration prefers local by default. Users with a slow or low-quality local model can disable just elaboration via: [local_model] endpoint = "..." prefer_for_elaborate = false without giving up the runner or the classifier path. Implementation: - Server gains an optional *llm.Client field via SetLLM (matches the existing SetNotifier/SetWorkspaceRoot setter pattern, no NewServer signature break). - elaborateWithLocal() reuses buildElaboratePrompt verbatim and asks for response_format=json_object so we skip markdown-fence cleanup. - handleElaborateTask reorders try chain; existing Claude-first behavior is preserved exactly when SetLLM is not called. - LocalModel.UseForElaborate() encapsulates the default-true gating with a *bool so explicit-false survives TOML parse. Tests: - elaborateWithLocal: parses valid response, errors on nil client, errors on bad JSON. - handler: local preferred when wired; falls back to claude when local fails; unchanged behavior when no LLM is configured. - config: UseForElaborate gating across empty/default/explicit-true/ explicit-false cases. Pre-existing test failures noted in docs/plans/local-oss-runner.md (post-epic cleanup): TestGeminiLogs_ParsedCorrectly returns 404 for gemini execution log fetch — predates this change. Plan: docs/plans/local-oss-runner.md. https://claude.ai/code/session_017Edeq947TpSm1vQTxMhi1J
Diffstat (limited to 'internal/config/config.go')
-rw-r--r--internal/config/config.go33
1 files changed, 25 insertions, 8 deletions
diff --git a/internal/config/config.go b/internal/config/config.go
index 7f87391..5801239 100644
--- a/internal/config/config.go
+++ b/internal/config/config.go
@@ -16,15 +16,32 @@ type Project struct {
}
// LocalModel configures an OpenAI-compatible local LLM endpoint used for
-// internal helpers (classifier, future elaboration/summarization) and as the
-// backend for the "local" runner. If Endpoint is empty, the LocalRunner is
-// not registered and the classifier falls back to the Gemini CLI.
+// internal helpers (classifier, elaboration, future summarization) and as
+// the backend for the "local" runner. If Endpoint is empty, the LocalRunner
+// is not registered and the classifier falls back to the Gemini CLI.
+//
+// PreferForElaborate gates whether the API server's elaboration handler
+// uses this client. It defaults to true when Endpoint is set; users with a
+// slow or low-quality local model can disable it.
type LocalModel struct {
- Endpoint string `toml:"endpoint"` // e.g. "http://localhost:11434/v1"
- Model string `toml:"model"` // e.g. "llama3.1:8b"
- TimeoutSeconds int `toml:"timeout_seconds"` // default 60
- DefaultTemperature float64 `toml:"default_temperature"` // default 0.2
- APIKey string `toml:"api_key"` // optional bearer token
+ Endpoint string `toml:"endpoint"` // e.g. "http://localhost:11434/v1"
+ Model string `toml:"model"` // e.g. "llama3.1:8b"
+ TimeoutSeconds int `toml:"timeout_seconds"` // default 60
+ DefaultTemperature float64 `toml:"default_temperature"` // default 0.2
+ APIKey string `toml:"api_key"` // optional bearer token
+ PreferForElaborate *bool `toml:"prefer_for_elaborate"` // pointer so default-true survives parse
+}
+
+// UseForElaborate returns true when elaboration should try this local model
+// before falling back to Claude/Gemini. Default is true when Endpoint is set.
+func (m LocalModel) UseForElaborate() bool {
+ if m.Endpoint == "" {
+ return false
+ }
+ if m.PreferForElaborate == nil {
+ return true
+ }
+ return *m.PreferForElaborate
}
type Config struct {