summaryrefslogtreecommitdiff
path: root/docs/adr/004-concurrent-data-fetching.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/adr/004-concurrent-data-fetching.md')
-rw-r--r--docs/adr/004-concurrent-data-fetching.md104
1 files changed, 104 insertions, 0 deletions
diff --git a/docs/adr/004-concurrent-data-fetching.md b/docs/adr/004-concurrent-data-fetching.md
new file mode 100644
index 0000000..c7238f8
--- /dev/null
+++ b/docs/adr/004-concurrent-data-fetching.md
@@ -0,0 +1,104 @@
+# ADR 004: Concurrent Data Fetching with Graceful Degradation
+
+## Status
+Accepted
+
+## Context
+The dashboard fetches data from 5+ external APIs on each page load:
+- Todoist (tasks)
+- Trello (boards, cards)
+- PlanToEat (meals, shopping)
+- Google Calendar (events)
+- Google Tasks (tasks)
+
+Sequential fetching would be unacceptably slow (5+ seconds). However, concurrent fetching introduces:
+- Rate limiting concerns (especially Trello)
+- Partial failure scenarios
+- Data race risks
+
+## Decision
+Implement **parallel fetching with semaphore-limited concurrency** and **cache-first fallback** for graceful degradation.
+
+### Technical Details:
+
+**Parallel Aggregation (`internal/handlers/handlers.go`):**
+```go
+func (h *Handler) aggregateData(ctx context.Context) (*DashboardData, error) {
+ var wg sync.WaitGroup
+ var mu sync.Mutex
+ data := &DashboardData{}
+
+ // Launch goroutines for each data source
+ wg.Add(4)
+ go func() { defer wg.Done(); fetchTasks(ctx, &mu, data) }()
+ go func() { defer wg.Done(); fetchBoards(ctx, &mu, data) }()
+ go func() { defer wg.Done(); fetchMeals(ctx, &mu, data) }()
+ go func() { defer wg.Done(); fetchCalendar(ctx, &mu, data) }()
+ wg.Wait()
+
+ return data, nil
+}
+```
+
+**Semaphore for Trello (`internal/api/trello.go`):**
+```go
+const maxConcurrentRequests = 5
+
+func (c *TrelloClient) GetBoardsWithCards(ctx context.Context) ([]Board, error) {
+ sem := make(chan struct{}, maxConcurrentRequests)
+ // ... limit concurrent board fetches
+}
+```
+
+**Cache-First Fallback:**
+```go
+func (h *Handler) fetchWithFallback(ctx context.Context, fetch func() error, getCache func() (T, error)) T {
+ if err := fetch(); err != nil {
+ log.Printf("Warning: API fetch failed, using cache: %v", err)
+ cached, _ := getCache()
+ return cached
+ }
+ return freshData
+}
+```
+
+**Error Collection:**
+- Errors are collected in `data.Errors` slice
+- UI displays warning banner if any source failed
+- Page still renders with available data
+
+## Consequences
+
+**Pros:**
+- Fast page loads (parallel fetching)
+- Resilient to individual API failures
+- Respects rate limits (semaphore pattern)
+- Users see cached data rather than error page
+
+**Cons:**
+- Complexity in error handling
+- Cache fallback may show stale data without clear indication
+- Debugging requires checking multiple error sources
+- Mutex needed for shared data structure
+
+## Alternatives Considered
+
+**Option A: Sequential fetching**
+- Rejected: 5+ second page loads unacceptable
+
+**Option B: Fail-fast on any error**
+- Rejected: One flaky API shouldn't break entire dashboard
+
+**Option C: Background sync with eventual consistency**
+- Rejected: Adds complexity, users expect fresh data on load
+
+## Implementation Notes
+
+**Race Condition Prevention:**
+- All goroutines write to different fields OR use mutex
+- Trello client uses mutex for `boards[i]` slice element writes
+- Run `go test -race ./...` to verify
+
+**Monitoring:**
+- Log all API failures with `Warning:` prefix
+- Consider adding metrics for cache hit/miss rates