diff options
| -rw-r--r-- | docs/detection-findings.md | 91 |
1 files changed, 62 insertions, 29 deletions
diff --git a/docs/detection-findings.md b/docs/detection-findings.md index 708400f..cacb2f0 100644 --- a/docs/detection-findings.md +++ b/docs/detection-findings.md @@ -9,15 +9,19 @@ The prototype is the right tool for this and the only one: it can hold a whole c plot correlations, and sweep parameters in seconds. Do not tune the constants from the app. **Update, later the same day:** the leading hypothesis below (the 8-second reference window -being too wide for a ~6-second sting) is disproven. See "What the desktop session actually -found" -- the production algorithm, run against real Up First audio at its real defaults, -confirms cleanly on every pair tested, both ends. The mystery is now sharper, not solved: the -identical algorithm works on desktop and reportedly found nothing on the phone. That gap is very -unlikely to be in `DEFAULT_REFERENCE_SECONDS`, `MIN_SCORE`, `CONFIRM_THRESHOLD` or the resampler --- all four were checked directly -- which points at something specific to the on-device run -itself (which episodes, in what order, whether each `FingerprintWorker` run actually completed) -rather than at the signal-processing approach. Diagnosing that needs on-device state, which this -sandbox cannot reach; see "What to check on the phone" at the end. +being too wide for a ~6-second sting) is disproven -- see "What the desktop session actually +found." The production algorithm, run against real Up First audio at its real defaults, confirms +cleanly on every pair tested, both ends. + +**Second update, the phone reconnected the same day:** pulled the live database and WorkManager's +own execution history straight off it (read-only, via `run-as`). The pipeline is not broken -- +**Cruising the Waves Podcast reached a real Tentative candidate (confirmedCount=2) today.** Up +First specifically is stuck: its `show_fingerprints` row hasn't updated since its *first* episode +was ever processed, even though a second real episode was downloaded, played and decoded +successfully (by a different worker) the next day. `FingerprintUpdater` upserts on every outcome, +match or not, so that frozen timestamp means the second episode's comparison never ran at all -- +not a score that failed. See "What the phone actually showed" below for the detail and "What to +check on the phone" for the one live test that would settle it. --- @@ -127,26 +131,55 @@ one. --- -## What to check on the phone - -Everything above rules out the algorithm and its tuned constants as an explanation, on real Up -First audio, at the app's real settings. That leaves the actual on-device run as the place to -look, and none of it can be done from this sandbox: - -- **What did the three downloaded episodes actually decode to?** A quick temporary log of - `clip.samples.size` and a couple of sample values in `FingerprintUpdater.process` would show - whether `MediaCodecClipSource` produced a full ~240s clip each time, or something truncated -- - a decode that silently stops early would explain "nothing found" without any correlation math - being wrong. -- **Did all three `FingerprintWorker` runs actually complete and persist?** `adb shell` + - `sqlite3` against `show_fingerprints` after each download would show `confirmedCount` climbing - toward 3 (or not) and whether `pendingEpisodeId` ever got set at all -- distinguishing "never - got past Waiting" from "got to Tentative and stalled," which "Nothing found yet" in the UI - cannot. -- **Were the three episodes actually three *different* episodes?** `IntroOutroDetector.check`'s - dedup guard silently no-ops on an id it has already seen (`lastConfirmedEpisodeId` or - `pendingEpisodeId`) -- worth ruling out that a re-queued or re-triggered download fed the same - episode through twice. +## What the phone actually showed (2026-09-29, later) + +The phone reconnected mid-session. Pulled `listen.db` (via `run-as` + `adb exec-out cat`, no +device modification) and `no_backup/androidx.work.workdb` -- WorkManager's own execution +history, which nothing in this app writes to but which turned out to answer the question the app +state alone couldn't. + +**The pipeline is not broken.** `show_fingerprints` for **Cruising the Waves Podcast** shows +`kind=INTRO, confirmedCount=2, referenceClipPcm` populated -- a real Tentative candidate, found +today, from an ordinary download. Something *does* reach Tentative on this exact build, on this +exact phone. The mechanism works in general; Up First specifically is the anomaly, not Tier 3. + +**Up First's `show_fingerprints` row is frozen at its very first episode.** Both INTRO and OUTRO +show `confirmedCount=0`, `referenceClipPcm=NULL`, `pendingEpisodeId` pointing at one single +episode ("Powerful storms hit the US..."), and `updatedAtMs = 2026-09-27T02:19` -- the moment +that first episode was processed. A second real episode ("Can clergy candidates help Democrats +win in red states?") was downloaded, played and swept by retention the *next day*, and its +`midrollCheckedAtMs` is set, proving `MidrollWorker`'s `decodeFull` read that file successfully +that day. But `show_fingerprints.updatedAtMs` never moved again. + +That is the specific, damning fact: `FingerprintUpdater.processEnd` calls `fingerprints.upsert(...)` +on **every** outcome -- a confirmed match, a failed match, all of it -- so if a comparison +involving the second episode had run at all, this row would show a later timestamp regardless of +whether it found anything. It doesn't. **The second episode's `FingerprintWorker` run either was +never enqueued, or `decodeHead`/`decodeTail` both returned null for it** -- and the latter looks +unlikely given `decodeFull` (the same underlying decode, just a different window) succeeded on +the identical file later that day. `EpisodeDownloader.downloadOverHttp` was read end to end look +for a race between "downloaded" and "playable" -- there isn't one, it's a fully blocking +download-to-`.part`-then-atomic-rename before `onDownloaded` fires, so a half-written file is not +the likely explanation either. + +WorkManager prunes completed `WorkSpec` rows after roughly a week, so the actual historical +`FingerprintWorker` run (if any) for that second episode is already gone -- only today's runs and +the next periodic `MidrollWorker` firing were still in the table. This is now a live-observation +question, not an archaeology one. + +## What to check on the phone (updated -- narrower than the original three) + +- **Trigger a fresh Up First download and watch `androidx.work.workdb` and `show_fingerprints` + immediately after**, the same way this session just did. If a `FingerprintWorker` `WorkSpec` + row appears for the new episode's id and reaches `SUCCEEDED`, but `show_fingerprints.updatedAtMs` + still doesn't move, that is a real bug in `processEnd`/the upsert path, reproduced live. If no + `WorkSpec` row appears at all, `onDownloaded` isn't firing for this show specifically -- check + whether Up First episodes are reaching playback through a route other than + `EpisodeDownloader.download()`. +- **If a `FingerprintWorker` row does appear and reaches SUCCEEDED with no DB change**, add a + temporary log of `clip.samples.size` in `FingerprintUpdater.process` for exactly this case -- + the one thing still not directly observed is whether `decodeHead`/`decodeTail` returned null or + an empty/garbage clip for that specific file. ## Experiments, in order (updated) |