diff options
Diffstat (limited to 'CLAUDE.md')
| -rw-r--r-- | CLAUDE.md | 81 |
1 files changed, 42 insertions, 39 deletions
@@ -146,45 +146,40 @@ downloaded at once, Facebook videos are supported, the library ranks shows by th and opt-in skipping of breaks the publisher marked `toc: false`. Nothing is guessed from chapter titles. -**Tier 3 skip detection — audio fingerprinting — is built, in two parts.** Both were desktop- -prototyped against real episodes before any Android code, per the design plan's own instruction. - -- **Intro/outro** (`IntroOutroDetector`, `AudioCorrelation`, `MediaCodecClipSource`, - `FingerprintUpdater`): one fixed candidate per show per end, needing 3+ independently-fetched - episodes to agree before it is trusted -- the confirmation threshold exists because two episodes - fetched close together can agree on a segment that is a dynamically-inserted ad, not the show's - real intro (found prototyping against Darknet Diaries). Triggered per download via - `FingerprintWorker`. -- **Midroll ad breaks** (`MidrollScanner`, `MidrollDetector`, `MidrollUpdater`): a show can have - several distinct midroll breaks, at a different timestamp in every episode, so this scans a - whole episode body rather than a fixed head/tail window -- `MidrollScanner.findSharedSegments` - finds every segment two episodes share, at whatever position each happens to have it, and - `MidrollUpdater` composes a coarse (decimated) pass with a native-resolution verify so that scan - is fast enough to run on a phone. Prototyping this against Darknet Diaries found a real recurring - segment (not the fixed intro) independently present across episodes nine years apart. Runs - nightly while charging (`MidrollWorker`), not per-download: a full-body scan is too expensive for - that, and unlike a download it never touches the network. No per-show toggle, same as - intro/outro -- the confirmation threshold is the safety mechanism. - -**Both halves have now run on a device**, and doing so found a blocking defect the unit tests -could never have caught. `show_fingerprints.pendingClipPcm` held a whole decoded head/tail -window -- 240 seconds at 8kHz is 3.84MB as 16-bit PCM -- and Android reads every Room row -through a CursorWindow, which refuses a row over 2MB *on the way out*. The insert succeeded; -the next read threw `SQLiteBlobTooBigException`. Intro/outro detection therefore crashed the -app the moment a show's second episode was fingerprinted, and could never have worked as -shipped. Fixed in migration 9 -> 10 by storing only `pendingEpisodeId` and re-decoding on -demand -- the pattern `midrollPendingEpisodeId` already used for the same reason. - -`StoredClip` now owns that limit, `MidrollUpdater` caps a candidate segment at 60 seconds -before persisting it (a long sponsor block would otherwise hit the same wall), and -`StoredClipSizeTest` pins both against the limit so the class of bug cannot come back quietly. - -What is proven on hardware: the MediaCodec decode loop runs, `FingerprintWorker` completes -SUCCESS per download, and the re-decode path is exercised. What is **not** yet proven is that -detection actually confirms anything -- three downloaded Up First episodes left both ends still -"Nothing found yet". That may be correct (the confirmation threshold is deliberately strict) or -it may mean the 8-second reference window is too wide for a ~6-second sting; the desktop -prototype is the thing to compare against, and nobody has. +**Tier 3 skip detection — audio fingerprinting — was built on the phone, and is gone from it.** +Deleted 2026-11-04, after `ReachabilityTest` was widened to classes and found that none of it had +been reachable since detection moved to the server: `FingerprintWorker` and `MidrollWorker` went +then, and the twelve classes they drove stayed, shipped in every APK. + +What went: `IntroOutroDetector`, `AudioCorrelation`, `MidrollScanner`, `MidrollDetector`, +`MidrollUpdater`, `FingerprintUpdater`, `MediaCodecClipSource`, `Decimate`, `Fft`, `MonoResampler`, +`PcmCodec`, `StoredClip` and 67 unit tests. **The tag `on-device-detection` is the last commit +that has it** — `git checkout on-device-detection -- app/src/main/java/org/terst/listen/fingerprint/` +brings it all back, and the detection it implemented now lives in `server/internal/skips` and +`server/internal/signature`, prototyped and measured against real audio there instead. + +What stayed, and why, because none of it is detection: `SkipSummary` words the readout, +`BreakTrack` and `BreakPopup` draw and explain the marks on the episode timeline, and +`MidrollClipStore` shrank to `deleteAllFor` alone — a phone that ran the old version still has +clip files on disk, and unsubscribing is the only thing that clears them. + +**The database was deliberately not touched.** `show_fingerprints`, `show_midrolls` and +`episode_midrolls` are the server's columns now — `ServerSkips` writes its answers into exactly the +columns the on-device detector used to, which is why playback needed no changes when detection +moved. No migration, no schema change, nothing for a real library to go wrong over. + +Two things the deletion left standing that a future session should look at rather than assume: + +- **The per-show detection readout has nothing feeding it.** `ShowSettingsViewModel.loadDetection` + reads `ShowFingerprintDao.get` and `MidrollDao.getCandidates`, and nothing has written a + *candidate* row since detection moved: the server writes per-episode ranges and offsets, not the + confirmation state the readout describes. So "2 of 3 episodes agree" can never appear, and a + phone that never ran the old version shows "nothing found yet" for every show while the server + is finding and publishing breaks. Predates this deletion; it is the same migration's doing. +- **`RetentionSweeper`'s pending-episode guard protects nothing.** It reads + `getAllPendingEpisodeIds` and `getAllMidrollPendingEpisodeIds` so retention cannot reclaim a file + mid-detection. Nothing sets a pending basis any more, so both are always empty — harmless, but a + stale row on an existing phone would pin that episode's file for ever. **ID3 `CHAP` frames are dropped, not deferred** — decided 2026-09-26 by measuring Peter's actual 61 subscriptions rather than guessing. Every feed's newest episode was fetched far enough to read @@ -213,6 +208,14 @@ by default and correct; it is simply waiting for a publisher who marks breaks. - **Auto-downloads are not retried.** A rule's download that fails is left FAILED rather than requeued: retrying forever against a 404 would burn the battery, and the failed row is visible in the episode list. Retry is the manual Download button. +> **The three bullets below are history, about code that no longer exists.** They describe the +> on-device detector deleted on 2026-11-04 (see the Tier 3 entry above and the `on-device-detection` +> tag). Kept because the measurements in them were expensive and the same questions will come back +> if audio fingerprinting is ever wanted again — but nothing here names a file in this repo any +> more, so do not go looking for `DEFAULT_REFERENCE_SECONDS`, `MidrollUpdater.discover` or +> `IntroOutroDetectorScaleTest`. The surviving instruction from all of it is the one that is not +> about the app at all: tune detection against real audio off-device, never from the phone. + - **Tier 3 had never confirmed anything, and there were two independent reasons, both now fixed.** Not the correlation algorithm -- a real production-algorithm simulation against real Up First audio confirmed cleanly at the app's actual defaults before any of this. Do not tune |