From 217c982403a105e986fb7ce13b922d9f08583632 Mon Sep 17 00:00:00 2001 From: Peter Stone Date: Mon, 5 Oct 2026 09:54:59 +0000 Subject: Move skip detection to a server, and find that it detects ads The server is Peter's design and it works: listend polls the phone's feed list, mirrors the newest 8 episodes per show, detects across all of them at once, and serves the offsets *and the audio they belong to*. Deployed on tungsten behind Apache at https://listen.terst.org. FingerprintWorker and MidrollWorker are deleted -- the battery drain is gone because the work is. Serving the audio is load-bearing. Dynamic ad insertion turned out to be on every episode: Grass Apes declares 22:05 and the server was served 28:18. An offset measured on one copy is minutes wrong for another, so the phone has to play the bytes the detector examined. The offsets land in the same columns the on-device detector wrote, so playback needed no changes and there is no migration. **The headline finding is a failure, and it is proved rather than suspected.** Detection reported the intro confirmed on 8 of 8 Radiolab episodes and passed its self-check. It had locked onto an advertisement. The chosen segment was searched for in an ad-free copy of the same episode fetched hours earlier: r = 0.0621. It is in the ad break, not the programme. The confirmation threshold assumes three *independently fetched* episodes. Mirroring 8 within one minute from the same ad server makes them maximally dependent -- whatever campaign is running is in all of them, identical -- and SelfCheck cannot catch it, because the segments genuinely are the same audio. Moving to a server fixed the cost problem and broke the independence the safety mechanism rests on. The published offsets are head-window matches, so they happen to skip pre-roll advertising, which is useful by accident and will move whenever the campaign rotates. It is not intro detection and is not described as such anywhere. The fix, not made: record when each episode was fetched and require the agreeing episodes to span more than one fetch occasion. Reasoning, and an argument for detecting the ads directly instead, in docs/detection-findings.md. Also recorded there: two tooling traps that each cost real time. `android-ui tap` can land on the soft keyboard, because the dump reports layout coordinates the IME does not move -- a Save button and a text field both appeared tapped and neither fired. And the server's token is alphanumeric now because `adb shell input text` ate the leading "-" of a base64url one and produced a silent 401 that looked like broken wiring. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B --- CLAUDE.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) (limited to 'CLAUDE.md') diff --git a/CLAUDE.md b/CLAUDE.md index 5a7d1ae..807a35c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -266,6 +266,46 @@ by default and correct; it is simply waiting for a publisher who marks breaks. seeks approximate on a VBR file, and the playback position is the crown jewel. Full numbers in `docs/detection-findings.md`. +- **Detection runs on a server now, and `server/` is part of this repo.** Go, deployed on + tungsten as `listend.service`, behind Apache at https://listen.terst.org with a Let's Encrypt + cert. Data in `/site/listen.terst.org/data` (`state.json`, `audio/`, `token`). It polls the + phone's feed list, mirrors the newest 8 episodes per show, detects across all of them at once, + and serves the offsets *and the audio they belong to*. `FingerprintWorker` and `MidrollWorker` + are gone from the app. + + **Serving the audio is load-bearing, not a convenience.** Dynamic ad insertion is on every + episode: Radiolab's Grass Apes declares 22:05 and the server was served 28:18. An offset + measured on one copy is minutes wrong for another, so the phone has to play the bytes the + detector examined. `EpisodeDownloader` prefers the server and falls back to the publisher -- + the fallback matters, because skips improve playback rather than being a precondition for it. + + **The offsets go into the same columns the on-device detector wrote**, which is why playback + needed no changes: `effectiveIntroMs`, `effectiveOutroMs` and `PlayheadPolicy` already read + them. No migration either. + +- **The confirmation threshold does not survive simultaneous mirroring, and currently detects + ads rather than intros.** Proved, not suspected: the segment confirmed on 8 of 8 Radiolab + episodes was searched for in an ad-free copy of the same episode fetched hours earlier and + scored **r = 0.0621**. It is in the ad break, not the programme. Mirroring 8 episodes within + one minute from the same ad server makes them maximally dependent, so whatever campaign is + running is in all of them, identical -- and `SelfCheck` cannot catch it, because the segments + genuinely *are* the same audio. Do not describe the current offsets as intro detection. The + fix is to record fetch times and require the agreeing episodes to span more than one fetch + occasion; full reasoning in `docs/detection-findings.md`. + +- **`android-ui tap` can land on the soft keyboard.** The dump reports the layout's own + coordinates, which the IME does not move, so an element listed at y=1643 can be underneath the + keyboard -- taps on it silently do nothing, which reads exactly like a broken handler. Found + the hard way: a Save button and a text field both appeared tapped and neither fired. Check + `adb shell dumpsys input_method | grep mInputShown` and dismiss with `input keyevent + KEYCODE_BACK` (not `KEYCODE_ESCAPE`, which does not close it) before tapping anything low on a + screen that has a text field. + +- **The server's API token is alphanumeric on purpose.** It is typed into a phone by hand, and + the first generated token was base64url -- whose leading `-` was eaten by `adb shell input + text` treating it as a flag, producing a silent 401 that looked like broken wiring. The + alphabet also omits l, I, 1, O and 0. + - **Android Auto is verified in a real car**, as of 2026-09-28. It appears, and connects. The notification Peter saw on first connect was Android Auto's own new-media-app announcement, not Listen: the only notification this app can raise is the Media3 transport control, which -- cgit v1.2.3