diff options
Diffstat (limited to 'issues/phase4_step2_research_sync.md')
| -rw-r--r-- | issues/phase4_step2_research_sync.md | 124 |
1 files changed, 0 insertions, 124 deletions
diff --git a/issues/phase4_step2_research_sync.md b/issues/phase4_step2_research_sync.md deleted file mode 100644 index d690e6f..0000000 --- a/issues/phase4_step2_research_sync.md +++ /dev/null @@ -1,124 +0,0 @@ -# Phase 4 Step 2: Research Efficient Sync - -## Research Findings - -### Todoist Sync API - -**Test Results (from `research_test.go`):** -- Full sync response: **17,502 bytes** -- Incremental sync response: **179 bytes** (99% reduction!) -- `sync_token` successfully enables incremental updates - -**Response Structure:** -```json -{ - "sync_token": "...", - "full_sync": true/false, - "items": [...] // tasks -} -``` - -**Item (Task) Fields Available:** -- `id`, `content`, `description`, `project_id` -- `checked` (completion status) -- `is_deleted` (for handling deletions) -- `updated_at` (for tracking changes) -- `due` (with `date`, `is_recurring`, `string`) - -**Implementation Approach:** -1. Store `sync_token` in database (new table or cache_metadata) -2. On refresh, call Sync API with stored token -3. Apply incremental changes to local database: - - Update modified tasks - - Delete tasks where `is_deleted=true` - - Mark tasks where `checked=true` as completed -4. If `full_sync=true` in response, clear and rebuild local state - -### Trello API Optimization - -**Test Results:** -- Full boards response: **48,228 bytes** (all fields) -- Limited fields (id,name): **462 bytes** (99% reduction!) -- Single board with embedded cards/lists: **10,452 bytes** - -**Optimization Strategies:** - -1. **Field Filtering** (immediate win): - ``` - /boards?fields=id,name,dateLastActivity - /boards/{id}/cards?fields=id,name,idList,due,url - ``` - -2. **Batch Requests** (reduce API calls): - ``` - /boards/{id}?cards=visible&card_fields=id,name,idList,due,url&lists=open&list_fields=id,name - ``` - - Fetches board + cards + lists in single request - -3. **`dateLastActivity` Caching**: - - Store `dateLastActivity` for each board - - On refresh, fetch boards with minimal fields - - Only fetch cards for boards where `dateLastActivity` changed - - Note: `since` parameter exists but returns full data (not truly incremental) - -### Architecture Implications - -**Current Architecture:** -- Cache-aside pattern: Check cache -> Miss -> Fetch API -> Store in cache -- Problem: Full data fetch on every cache miss - -**Proposed Architecture:** -- **Sync Engine Pattern**: - - SQLite is the source of truth - - Sync process updates SQLite incrementally - - Read operations always hit SQLite (fast) - -**Migration Path:** -1. Phase A: Add field filtering to Trello (immediate benefit, no architecture change) -2. Phase B: Implement Todoist Sync API with sync_token storage -3. Phase C: Add dateLastActivity tracking for Trello boards - -## Implementation Plan - -### Phase A: Trello Field Optimization (Low effort, High impact) - -1. Update `GetBoards` to use `fields=id,name,dateLastActivity` -2. Update `GetCards` to use `fields=id,name,idList,due,url,desc` -3. Consider batch endpoint for single-board-with-cards - -### Phase B: Todoist Incremental Sync (Medium effort, High impact) - -1. Add `sync_tokens` table: - ```sql - CREATE TABLE sync_tokens ( - service TEXT PRIMARY KEY, - token TEXT NOT NULL, - updated_at DATETIME DEFAULT CURRENT_TIMESTAMP - ); - ``` - -2. Create `SyncTasks` method in `TodoistClient`: - - Accept previous sync_token - - Return new sync_token + changed items - -3. Create `ApplyTaskChanges` method in `Store`: - - Handle updates, deletions, completions - - Update sync_token after successful apply - -4. Update `fetchTasks` to use sync when token exists - -### Phase C: Trello Smart Refresh (Medium effort, Medium impact) - -1. Store `dateLastActivity` per board in database -2. On refresh: - - Fetch boards with minimal fields - - Compare `dateLastActivity` with stored values - - Only fetch cards for changed boards - -## Recommendation - -**Start with Phase A** - simple field filtering provides immediate 99% reduction in Trello response size with minimal code changes. - -**Then Phase B** - Todoist Sync API provides the best efficiency gains (99% reduction) and is well-documented. - -**Phase C is optional** - Trello doesn't have true incremental sync, so the benefit is limited to skipping unchanged boards. |
