diff options
| author | Claude <noreply@anthropic.com> | 2026-04-28 09:24:43 +0000 |
|---|---|---|
| committer | Claude <noreply@anthropic.com> | 2026-04-28 09:24:43 +0000 |
| commit | 0865afc43be562dbe14528e4299b9e213b54cc93 (patch) | |
| tree | 3ffb11207fb6b9866b5a2477bba7abe38964f83a /internal/executor/ratelimit.go | |
| parent | c2aa026f6ce1c9e216b99d74f294fc133d5fcddd (diff) | |
feat(executor): add LocalRunner and OpenAI-compat LLM client
Phase 1 of "local OSS models as agents" plan. Adds a third Runner
backed by any OpenAI-compatible HTTP server (Ollama, vLLM, LM Studio,
llama.cpp), and migrates the Gemini-CLI classifier to route through
the same client when configured.
Two-layer split: internal/llm.Client is the workhorse (HTTP, no Pool,
no DB) used directly by the classifier and any future internal helper
that needs cheap reasoning. internal/executor.LocalRunner is a thin
adapter implementing Runner for user-facing tasks. This avoids
Pool reentrancy/deadlock when sub-second internal calls fire from
inside Pool.execute().
Highlights:
- internal/retry: relocated runWithBackoff/IsRateLimitError/ParseRetryAfter
into a shared package reused by executor and llm.
- internal/llm: Chat (non-streaming) and ChatStream (SSE) over
/chat/completions with optional bearer auth, json_object response
format, retry on 429/503, Retry-After parsing.
- internal/executor/LocalRunner: streams deltas into stdout.log in the
same stream-json envelope ClaudeRunner emits, then writes one
consolidated assistant block plus a result terminator so existing
parsers (extractSummary, ParseChangestatFromOutput) work unchanged.
- internal/executor/Classifier: gains optional LLM field; uses
json_object response format (no markdown-fence cleanup needed).
Falls back to Gemini-CLI subprocess when LLM is nil.
- Pool.skipClassification: now skips only when the requested agent
type is registered, so unknown types still reach the load balancer.
- Storage: additive tokens_in/tokens_out ALTERs on executions; CLI
runners record cost_usd as before, LocalRunner records 0 + tokens.
- Config: [local_model] section (endpoint, model, timeout_seconds,
default_temperature, api_key). Empty endpoint = no LocalRunner
registered, classifier falls back to Gemini.
Pre-existing test issues fixed in passing:
- claude_test.go setupSandbox callsites updated to current signature.
- gemini_test.go TestParseGeminiStream skipped (asserts unimplemented
GeminiRunner stream-error parsing; tracked separately).
Plan: docs/plans/local-oss-runner.md.
https://claude.ai/code/session_017Edeq947TpSm1vQTxMhi1J
Diffstat (limited to 'internal/executor/ratelimit.go')
| -rw-r--r-- | internal/executor/ratelimit.go | 80 |
1 files changed, 3 insertions, 77 deletions
diff --git a/internal/executor/ratelimit.go b/internal/executor/ratelimit.go index 1f38a6d..109aa49 100644 --- a/internal/executor/ratelimit.go +++ b/internal/executor/ratelimit.go @@ -1,33 +1,9 @@ package executor -import ( - "context" - "fmt" - "regexp" - "strconv" - "strings" - "time" -) +import "strings" -var retryAfterRe = regexp.MustCompile(`(?i)retry[-_ ]after[:\s]+(\d+)`) - -const maxBackoffDelay = 5 * time.Minute - -// isRateLimitError returns true if err looks like a transient Claude API -// rate-limit that is worth retrying (e.g. per-minute/per-request throttle). -func isRateLimitError(err error) bool { - if err == nil { - return false - } - msg := strings.ToLower(err.Error()) - return strings.Contains(msg, "rate limit") || - strings.Contains(msg, "too many requests") || - strings.Contains(msg, "429") || - strings.Contains(msg, "overloaded") -} - -// isQuotaExhausted returns true if err indicates the 5-hour usage quota is -// fully exhausted. Unlike transient rate limits, these should not be retried. +// isQuotaExhausted returns true if err indicates the 5-hour Claude usage quota +// is fully exhausted. Unlike transient rate limits, these should not be retried. func isQuotaExhausted(err error) bool { if err == nil { return false @@ -39,53 +15,3 @@ func isQuotaExhausted(err error) bool { strings.Contains(msg, "rate limit reached (rejected)") || strings.Contains(msg, "status: rejected") } - -// parseRetryAfter extracts a Retry-After duration from an error message. -// Returns 0 if no retry-after value is found. -func parseRetryAfter(msg string) time.Duration { - m := retryAfterRe.FindStringSubmatch(msg) - if m == nil { - return 0 - } - secs, err := strconv.Atoi(m[1]) - if err != nil || secs <= 0 { - return 0 - } - return time.Duration(secs) * time.Second -} - -// runWithBackoff calls fn repeatedly on rate-limit errors, using exponential backoff. -// maxRetries is the max number of retry attempts (not counting the initial call). -// baseDelay is the initial backoff duration (doubled each retry). -func runWithBackoff(ctx context.Context, maxRetries int, baseDelay time.Duration, fn func() error) error { - var lastErr error - for attempt := 0; attempt <= maxRetries; attempt++ { - lastErr = fn() - if lastErr == nil { - return nil - } - if !isRateLimitError(lastErr) { - return lastErr - } - if attempt == maxRetries { - break - } - - // Compute exponential backoff delay. - delay := baseDelay * (1 << attempt) - if delay > maxBackoffDelay { - delay = maxBackoffDelay - } - // Use Retry-After header value if present. - if ra := parseRetryAfter(lastErr.Error()); ra > 0 { - delay = ra - } - - select { - case <-ctx.Done(): - return fmt.Errorf("context cancelled during rate-limit backoff: %w", ctx.Err()) - case <-time.After(delay): - } - } - return lastErr -} |
