diff options
Diffstat (limited to 'docs/adr/004-concurrent-data-fetching.md')
| -rw-r--r-- | docs/adr/004-concurrent-data-fetching.md | 104 |
1 files changed, 104 insertions, 0 deletions
diff --git a/docs/adr/004-concurrent-data-fetching.md b/docs/adr/004-concurrent-data-fetching.md new file mode 100644 index 0000000..c7238f8 --- /dev/null +++ b/docs/adr/004-concurrent-data-fetching.md @@ -0,0 +1,104 @@ +# ADR 004: Concurrent Data Fetching with Graceful Degradation + +## Status +Accepted + +## Context +The dashboard fetches data from 5+ external APIs on each page load: +- Todoist (tasks) +- Trello (boards, cards) +- PlanToEat (meals, shopping) +- Google Calendar (events) +- Google Tasks (tasks) + +Sequential fetching would be unacceptably slow (5+ seconds). However, concurrent fetching introduces: +- Rate limiting concerns (especially Trello) +- Partial failure scenarios +- Data race risks + +## Decision +Implement **parallel fetching with semaphore-limited concurrency** and **cache-first fallback** for graceful degradation. + +### Technical Details: + +**Parallel Aggregation (`internal/handlers/handlers.go`):** +```go +func (h *Handler) aggregateData(ctx context.Context) (*DashboardData, error) { + var wg sync.WaitGroup + var mu sync.Mutex + data := &DashboardData{} + + // Launch goroutines for each data source + wg.Add(4) + go func() { defer wg.Done(); fetchTasks(ctx, &mu, data) }() + go func() { defer wg.Done(); fetchBoards(ctx, &mu, data) }() + go func() { defer wg.Done(); fetchMeals(ctx, &mu, data) }() + go func() { defer wg.Done(); fetchCalendar(ctx, &mu, data) }() + wg.Wait() + + return data, nil +} +``` + +**Semaphore for Trello (`internal/api/trello.go`):** +```go +const maxConcurrentRequests = 5 + +func (c *TrelloClient) GetBoardsWithCards(ctx context.Context) ([]Board, error) { + sem := make(chan struct{}, maxConcurrentRequests) + // ... limit concurrent board fetches +} +``` + +**Cache-First Fallback:** +```go +func (h *Handler) fetchWithFallback(ctx context.Context, fetch func() error, getCache func() (T, error)) T { + if err := fetch(); err != nil { + log.Printf("Warning: API fetch failed, using cache: %v", err) + cached, _ := getCache() + return cached + } + return freshData +} +``` + +**Error Collection:** +- Errors are collected in `data.Errors` slice +- UI displays warning banner if any source failed +- Page still renders with available data + +## Consequences + +**Pros:** +- Fast page loads (parallel fetching) +- Resilient to individual API failures +- Respects rate limits (semaphore pattern) +- Users see cached data rather than error page + +**Cons:** +- Complexity in error handling +- Cache fallback may show stale data without clear indication +- Debugging requires checking multiple error sources +- Mutex needed for shared data structure + +## Alternatives Considered + +**Option A: Sequential fetching** +- Rejected: 5+ second page loads unacceptable + +**Option B: Fail-fast on any error** +- Rejected: One flaky API shouldn't break entire dashboard + +**Option C: Background sync with eventual consistency** +- Rejected: Adds complexity, users expect fresh data on load + +## Implementation Notes + +**Race Condition Prevention:** +- All goroutines write to different fields OR use mutex +- Trello client uses mutex for `boards[i]` slice element writes +- Run `go test -race ./...` to verify + +**Monitoring:** +- Log all API failures with `Warning:` prefix +- Consider adding metrics for cache hit/miss rates |
