1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
# ADR 004: Concurrent Data Fetching with Graceful Degradation
## Status
Accepted
## Context
The dashboard fetches data from 5+ external APIs on each page load:
- Todoist (tasks)
- Trello (boards, cards)
- PlanToEat (meals, shopping)
- Google Calendar (events)
- Google Tasks (tasks)
Sequential fetching would be unacceptably slow (5+ seconds). However, concurrent fetching introduces:
- Rate limiting concerns (especially Trello)
- Partial failure scenarios
- Data race risks
## Decision
Implement **parallel fetching with semaphore-limited concurrency** and **cache-first fallback** for graceful degradation.
### Technical Details:
**Parallel Aggregation (`internal/handlers/handlers.go`):**
```go
func (h *Handler) aggregateData(ctx context.Context) (*DashboardData, error) {
var wg sync.WaitGroup
var mu sync.Mutex
data := &DashboardData{}
// Launch goroutines for each data source
wg.Add(4)
go func() { defer wg.Done(); fetchTasks(ctx, &mu, data) }()
go func() { defer wg.Done(); fetchBoards(ctx, &mu, data) }()
go func() { defer wg.Done(); fetchMeals(ctx, &mu, data) }()
go func() { defer wg.Done(); fetchCalendar(ctx, &mu, data) }()
wg.Wait()
return data, nil
}
```
**Semaphore for Trello (`internal/api/trello.go`):**
```go
const maxConcurrentRequests = 5
func (c *TrelloClient) GetBoardsWithCards(ctx context.Context) ([]Board, error) {
sem := make(chan struct{}, maxConcurrentRequests)
// ... limit concurrent board fetches
}
```
**Cache-First Fallback:**
```go
func (h *Handler) fetchWithFallback(ctx context.Context, fetch func() error, getCache func() (T, error)) T {
if err := fetch(); err != nil {
log.Printf("Warning: API fetch failed, using cache: %v", err)
cached, _ := getCache()
return cached
}
return freshData
}
```
**Error Collection:**
- Errors are collected in `data.Errors` slice
- UI displays warning banner if any source failed
- Page still renders with available data
## Consequences
**Pros:**
- Fast page loads (parallel fetching)
- Resilient to individual API failures
- Respects rate limits (semaphore pattern)
- Users see cached data rather than error page
**Cons:**
- Complexity in error handling
- Cache fallback may show stale data without clear indication
- Debugging requires checking multiple error sources
- Mutex needed for shared data structure
## Alternatives Considered
**Option A: Sequential fetching**
- Rejected: 5+ second page loads unacceptable
**Option B: Fail-fast on any error**
- Rejected: One flaky API shouldn't break entire dashboard
**Option C: Background sync with eventual consistency**
- Rejected: Adds complexity, users expect fresh data on load
## Implementation Notes
**Race Condition Prevention:**
- All goroutines write to different fields OR use mutex
- Trello client uses mutex for `boards[i]` slice element writes
- Run `go test -race ./...` to verify
**Monitoring:**
- Log all API failures with `Warning:` prefix
- Consider adding metrics for cache hit/miss rates
|
