diff options
Diffstat (limited to 'CLAUDE.md')
| -rw-r--r-- | CLAUDE.md | 64 |
1 files changed, 64 insertions, 0 deletions
@@ -347,6 +347,70 @@ by default and correct; it is simply waiting for a publisher who marks breaks. eaten). Both record the refusal, because "found nothing" and "refused what it found" are different states and conflating them is the readout mistake again. +- **Episodes have words now, and the words are the best ad detector in the project.** Two + sources, and they are not interchangeable. + + **A publisher's own `<podcast:transcript>`** is exact and free. Eleven of 44 feeds carry one. + It describes the publisher's *master*, so it is only usable where the show inserts nothing: + measured across every mirrored show that publishes one, the transcript tracks the **declared** + length every time, which means it aligns precisely when served and declared agree and is + minutes out of step precisely when they do not. `transcript.Fits` makes that call from the two + lengths rather than from the words. 38 taken, 19 declined. + + **Speech recognition** (whisper.cpp, `base.en`, installed at `/opt/whisper`) describes exactly + the bytes the phone will play. Measured, not estimated: **10.3x realtime on six threads, 793MB + peak** -- a 63-minute episode in 6m11s. The earlier guess of "roughly realtime" was wrong by an + order of magnitude. + + **A declined transcript is kept as the master, not thrown away.** Declined means "this copy + carries inserted advertising the master does not describe", which is precisely the episode + subtraction works on. + +- **`transcript.Inserted` finds advertising by subtraction, and it is a measurement rather than + a guess.** The master says what the programme is, the recognised copy says what was played, + and the difference is the advertising by definition. Across the 18 episodes holding a master + it accounts for **92% to 104%** of what the declared-versus-served length gap implies, finding + every pod. Reading ad copy (`transcript.Spot`) scores 0-29% on the same episodes, and only the + first is ever published. + + Four things it took measurement to get right, all of them documented in their own commits: + anchors name the *start* of a matching run (getting that wrong skipped the programme's last + sentence); a gap is partitioned line by line rather than refused when the master drifted + (which was losing a whole pod on three episodes); whisper's `[music]` notes are trimmed from + break edges but still shown; and a short line is judged on a three-word run, because "I'm Jack + Rhysider" comes back as "Resider" and one misheard word in four hid the host's own sentence + inside a break. + +- **Transcript breaks are published, and this is verified on a device end to end** (2026-10-31). + Heavyweight's "Jonathan + Death of an Artist", breaks at 0-1:45 and 13:32-15:16: downloaded + from the server, both rows landed in `episode_midrolls`, playback started past the pre-roll + (20 seconds of playing put the position at 2:03) and the post-roll skipped to the end, marked + the episode played and reset the position. + + They live in `store.Episode.WordBreaks`, **not** beside the audio scan's: `detectAds` rewrites + `Breaks` from scratch on every pass and would wipe them. `Episode.AllBreaks()` serves the + union, merging ranges that overlap or touch, with advertising winning where two disagree. + + **`transcript.Publishable` is the only thing that turns words into a seek.** It publishes + subtraction only, refuses anything for an episode with no length gap to check against, and + refuses a candidate set claiming more than 150% of what was inserted. + +- **Refetching a second copy is for slow-drip and leisure shows, not daily news.** Peter's steer, + and the numbers agree: a second fetch minutes after the first came back 20 seconds different + and subtraction found 21% of the advertising; another came back the same length to the second + and found none. Insertion varies the *amount* per request, not the campaign. Gated to episodes + four weeks old. + +- **Text shared across a show's episodes is the strongest unexploited signal, and it is free.** + Eight episodes mirrored within a minute carry the same campaign and different programmes, so + text appearing across several of them is the advertising -- the text analogue of the audio + scan, using transcripts already on disk. Prototyped against the live library: Heavyweight 111%, + Incels 105%, Ezra Klein 92%, Tangle 83% (against refetching's 21% on the same show), covering + 21 shows. It also reaches the ~81 episodes with no length gap at all, where ads are baked into + what the publisher serves and nothing else can see them. **It catches a show's own furniture + too** -- Tangle's welcome, its credits, "we'll be right back after this quick break" -- so it + needs the same intro/cue/outro/ad classification `skips.Classify` already does. Not built. + - **Intro and outro offsets are computed, logged, and deliberately not published.** They assume the agreeing episodes were fetched independently, and mirroring eight at once means they were not. Proved, not suspected: the segment confirmed on 8 of 8 Radiolab episodes was searched for |