summaryrefslogtreecommitdiff
path: root/CLAUDE.md
diff options
context:
space:
mode:
authorPeter Stone <thepeterstone@gmail.com>2026-10-06 07:31:53 +0000
committerPeter Stone <thepeterstone@gmail.com>2026-10-06 07:31:53 +0000
commit7847e13c3fab23edfda096396e4dbe01df3471c3 (patch)
treec4ba6d0b84b0abd58fed08e2ba09c6ee6f682ac5 /CLAUDE.md
parent217c982403a105e986fb7ce13b922d9f08583632 (diff)
Follow the data: detect the advertising, not the intro
The property that broke intro detection is the property that makes this work. Eight episodes mirrored within one minute are maximally dependent, because the ad server inserts whatever campaign is running into all of them identically. For intro detection that destroys the independence the confirmation threshold needs -- which is how it came to confirm an advertisement on 8 of 8 episodes. For ad detection it is the signal. And unlike intro detection it is checkable against a number the detector does not produce: the feed declares the ad-free length, the file served is longer, and the difference is the advertising. **Measured on the live deployment, Radiolab's eight mirrored episodes: the scan accounted for 53:29 of the 59:39 the length gap implies, with every individual episode between 71% and 105%.** That is the first time detection in this project has been checked against external ground truth and agreed with it. The old sliding search could never have done this -- one full-length correlation per window position, 199 seconds for a 240-second clip. BestLags computes the cross-correlation over every lag in a single FFT pair, which says where two recordings align; MatchedRuns then walks the extent of the agreement directly, window by window, with no transform. Sized against measurements rather than guesses: 2^22 is 1.69s, so a 70-minute episode decimated to 250Hz is about five seconds a pair. Silence is explicitly not agreement, or separate advertisements weld together across the pauses between them. Breaks publish as ranges into episode_midrolls, which MidrollSkip and PlayheadPolicy already read -- no playback changes and no migration. Intro and outro offsets are still computed and logged but no longer published, and the ad-derived ones already in the server's state were cleared. Two guards publish nothing rather than something wrong: an episode whose matches cover over 35% of it, and any single match over four minutes. A re-aired episode or a two-parter would otherwise be eaten. Both record the refusal, because "found nothing" and "refused what it found" are different states and conflating them is the readout mistake over again. Also three data-loss bugs, found by reading the journal after a restart rather than by any test. The subscription list -- the one thing the server cannot rediscover -- was persisted only when a whole poll cycle finished, which for 61 feeds is hours. A cycle re-applied its stale snapshot at the end, which would undo a push that arrived during it and then delete the audio of every show the snapshot omitted. And detection ignored cancellation, so systemd's 90-second stop SIGKILLed the process before the save-on-exit could run. All three fixed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B
Diffstat (limited to 'CLAUDE.md')
-rw-r--r--CLAUDE.md45
1 files changed, 36 insertions, 9 deletions
diff --git a/CLAUDE.md b/CLAUDE.md
index 807a35c..f0b6412 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -283,15 +283,42 @@ by default and correct; it is simply waiting for a publisher who marks breaks.
needed no changes: `effectiveIntroMs`, `effectiveOutroMs` and `PlayheadPolicy` already read
them. No migration either.
-- **The confirmation threshold does not survive simultaneous mirroring, and currently detects
- ads rather than intros.** Proved, not suspected: the segment confirmed on 8 of 8 Radiolab
- episodes was searched for in an ad-free copy of the same episode fetched hours earlier and
- scored **r = 0.0621**. It is in the ad break, not the programme. Mirroring 8 episodes within
- one minute from the same ad server makes them maximally dependent, so whatever campaign is
- running is in all of them, identical -- and `SelfCheck` cannot catch it, because the segments
- genuinely *are* the same audio. Do not describe the current offsets as intro detection. The
- fix is to record fetch times and require the agreeing episodes to span more than one fetch
- occasion; full reasoning in `docs/detection-findings.md`.
+- **The server detects the advertising, not the intro, and that is now deliberate.** The
+ property that broke intro detection is what makes ad detection work: eight episodes mirrored
+ within one minute share whatever ad campaign is running, identically. `detect.BestLags` finds
+ where two recordings align with one cross-correlation over every lag -- not one correlation
+ per window position, which is why whole episodes are now scannable at all -- and
+ `detect.MatchedRuns` walks the extent of the agreement directly.
+
+ **It is validated against a number the detector does not produce.** The feed declares the
+ ad-free length, the file is longer, and the difference is the advertising. On Radiolab's eight
+ mirrored episodes the scan accounted for **53:29 of the 59:39** implied, every episode between
+ 71% and 105%. First time in this project that detection has been checked against external
+ ground truth and agreed.
+
+ Breaks are published as ranges into `episode_midrolls`, which `MidrollSkip` and
+ `PlayheadPolicy` already read -- no playback changes, no migration. Two guards publish nothing
+ rather than something wrong: an episode whose matches cover over 35% of it, and any single
+ match over four minutes, are refused (a re-aired episode or a two-parter would otherwise be
+ eaten). Both record the refusal, because "found nothing" and "refused what it found" are
+ different states and conflating them is the readout mistake again.
+
+- **Intro and outro offsets are computed, logged, and deliberately not published.** They assume
+ the agreeing episodes were fetched independently, and mirroring eight at once means they were
+ not. Proved, not suspected: the segment confirmed on 8 of 8 Radiolab episodes was searched for
+ in an ad-free copy of the same episode fetched hours earlier and scored **r = 0.0621** -- it
+ was in the ad break. `SelfCheck` cannot catch that, because the segments genuinely *are* the
+ same audio. They become trustworthy once the agreeing episodes are required to span more than
+ one fetch occasion; until then do not publish them.
+
+- **The subscription list is the one thing the server cannot rediscover, and it was being lost.**
+ Three bugs, all found by reading the journal after a restart rather than by a test: the list
+ was persisted only when a whole poll cycle finished (hours, for 61 feeds); a cycle re-applied
+ its stale snapshot at the end, which would undo a push that arrived during it and delete the
+ audio of every show the snapshot omitted; and detection ignored cancellation, so systemd's
+ 90-second stop SIGKILLed the process before the save-on-exit could run. Now: saved on arrival,
+ saved after every feed, reconciled against the live list, cancellation honoured, and
+ `TimeoutStopSec=600`.
- **`android-ui tap` can land on the soft keyboard.** The dump reports the layout's own
coordinates, which the IME does not move, so an element listed at y=1643 can be underneath the