1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
|
# Phase 4 Step 2: Research Efficient Sync
## Research Findings
### Todoist Sync API
**Test Results (from `research_test.go`):**
- Full sync response: **17,502 bytes**
- Incremental sync response: **179 bytes** (99% reduction!)
- `sync_token` successfully enables incremental updates
**Response Structure:**
```json
{
"sync_token": "...",
"full_sync": true/false,
"items": [...] // tasks
}
```
**Item (Task) Fields Available:**
- `id`, `content`, `description`, `project_id`
- `checked` (completion status)
- `is_deleted` (for handling deletions)
- `updated_at` (for tracking changes)
- `due` (with `date`, `is_recurring`, `string`)
**Implementation Approach:**
1. Store `sync_token` in database (new table or cache_metadata)
2. On refresh, call Sync API with stored token
3. Apply incremental changes to local database:
- Update modified tasks
- Delete tasks where `is_deleted=true`
- Mark tasks where `checked=true` as completed
4. If `full_sync=true` in response, clear and rebuild local state
### Trello API Optimization
**Test Results:**
- Full boards response: **48,228 bytes** (all fields)
- Limited fields (id,name): **462 bytes** (99% reduction!)
- Single board with embedded cards/lists: **10,452 bytes**
**Optimization Strategies:**
1. **Field Filtering** (immediate win):
```
/boards?fields=id,name,dateLastActivity
/boards/{id}/cards?fields=id,name,idList,due,url
```
2. **Batch Requests** (reduce API calls):
```
/boards/{id}?cards=visible&card_fields=id,name,idList,due,url&lists=open&list_fields=id,name
```
- Fetches board + cards + lists in single request
3. **`dateLastActivity` Caching**:
- Store `dateLastActivity` for each board
- On refresh, fetch boards with minimal fields
- Only fetch cards for boards where `dateLastActivity` changed
- Note: `since` parameter exists but returns full data (not truly incremental)
### Architecture Implications
**Current Architecture:**
- Cache-aside pattern: Check cache -> Miss -> Fetch API -> Store in cache
- Problem: Full data fetch on every cache miss
**Proposed Architecture:**
- **Sync Engine Pattern**:
- SQLite is the source of truth
- Sync process updates SQLite incrementally
- Read operations always hit SQLite (fast)
**Migration Path:**
1. Phase A: Add field filtering to Trello (immediate benefit, no architecture change)
2. Phase B: Implement Todoist Sync API with sync_token storage
3. Phase C: Add dateLastActivity tracking for Trello boards
## Implementation Plan
### Phase A: Trello Field Optimization (Low effort, High impact)
1. Update `GetBoards` to use `fields=id,name,dateLastActivity`
2. Update `GetCards` to use `fields=id,name,idList,due,url,desc`
3. Consider batch endpoint for single-board-with-cards
### Phase B: Todoist Incremental Sync (Medium effort, High impact)
1. Add `sync_tokens` table:
```sql
CREATE TABLE sync_tokens (
service TEXT PRIMARY KEY,
token TEXT NOT NULL,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```
2. Create `SyncTasks` method in `TodoistClient`:
- Accept previous sync_token
- Return new sync_token + changed items
3. Create `ApplyTaskChanges` method in `Store`:
- Handle updates, deletions, completions
- Update sync_token after successful apply
4. Update `fetchTasks` to use sync when token exists
### Phase C: Trello Smart Refresh (Medium effort, Medium impact)
1. Store `dateLastActivity` per board in database
2. On refresh:
- Fetch boards with minimal fields
- Compare `dateLastActivity` with stored values
- Only fetch cards for changed boards
## Recommendation
**Start with Phase A** - simple field filtering provides immediate 99% reduction in Trello response size with minimal code changes.
**Then Phase B** - Todoist Sync API provides the best efficiency gains (99% reduction) and is well-documented.
**Phase C is optional** - Trello doesn't have true incremental sync, so the benefit is limited to skipping unchanged boards.
|
