diff options
| -rw-r--r-- | CLAUDE.md | 29 | ||||
| -rw-r--r-- | docs/detection-findings.md | 107 | ||||
| -rw-r--r-- | server/internal/transcript/publish.go | 23 |
3 files changed, 150 insertions, 9 deletions
@@ -41,8 +41,14 @@ in that literal order: 6. Run the full quality gate before committing. **If production code was written before or without a preceding failing test**, discard the diff and -redo it test-first. Do not backfill tests onto existing code — a test written against working code -confirms what the code does, not what it should do. +redo it test-first. Do not backfill tests onto existing code *as a substitute for that* — a test +written against working code confirms what the code does, not what it should do, so it cannot stand +in for a red step. + +The one case it is right: **pinning behaviour before moving it.** A characterisation test written +against the old location, confirmed green, then re-run after the move is what makes a refactor +provably behaviour-preserving — the "keeping it green" in step 5 needs a green to keep. Write it in +that order and say so; written afterwards it only proves the new code agrees with itself. **Scoping exception:** pure resource/config/manifest changes with no unit-testable behaviour don't get a red step; the static-analysis gate covers those. @@ -115,7 +121,9 @@ claims three separate times across these repos. **Phase 1 is complete.** Subscribe (share sheet, pasted URL, OPML import), refresh, download, queue, play, variable speed, sleep timer, retention, unsubscribe, Android Auto with a browsable -library and voice search. 207 unit tests plus instrumented tests covering the media browse tree. +library and voice search, with instrumented tests covering the media browse tree. (Counts are +deliberately not recorded here -- they went stale at 207 while the suite reached nine hundred. +`gradle-test` and `go-verify` report the current ones.) **Since Phase 2**, in response to a round of Peter's feedback: yt-dlp sources play (they are fetched first), playback errors are surfaced instead of being swallowed, anything queued is @@ -549,7 +557,11 @@ by default and correct; it is simply waiting for a publisher who marks breaks. playing episode's screen, and steps aside on that screen. Speed is a chip (`SpeedChoices.OFFERED` is 1.0/1.1/1.25/2.0); a show stored at 1.5 or 1.75 keeps playing at it, the chip just names it. Secondary actions and chapters are in the `⋮` menu, whose contents `EpisodeMenu` decides, and - the description shows ~two lines (`DescriptionPreview`) with the rest on tap. + the description shows about ten lines (`DescriptionPreview.PREVIEW_LINES`, with a character + budget beside it) and the rest on tap. It was two until 2026-11-01: that limit existed to + stop long notes pushing the transport off-screen, and the merge put the notes *below* the + transport, so the reason had already gone. What it actually did was cut at 140 characters + and leave the bottom three-fifths of the screen empty. Do not reinstate it. - **Framework adapters are excluded from coverage, as of 2026-09-26.** The gate had been overridden eight times, every one a class whose whole body is a call into ExoPlayer, Media3, @@ -615,6 +627,15 @@ by default and correct; it is simply waiting for a publisher who marks breaks. error colour through the dimming, because a feed failing to refresh looks exactly like a feed with nothing new. +- **Anything the app says out loud lives in a named file of its own, and is tested.** Not in the + composable that renders it: `SkipSummary`, `RuleSummary`, `FoundBySummary`, `WaitingForServerText`, + `RejectedTokenText`, `DescriptionPreview`, `SpeedChoices`, `TimeText` (`formatClock` for a + position, `formatDuration` for a length) and `EpisodeLabels` (`progressLabel`, the one line under + every episode title). Screens are excluded from coverage and verified on hardware, so wording left + inside one is wording nothing checks -- and a shared formatter inside a view means editing the + now-playing bar to change the transcript gutter, which is where `formatClock` actually was until + 2026-11-02. A new user-visible string either joins one of these or brings its own file. + - **What the skip detection has worked out is visible**, per show in the settings sheet and per episode on its detail screen. `SkipSummary` owns the wording and is tested, because the distinction that matters -- confirmed-and-acting versus still-gathering -- is the whole point, diff --git a/docs/detection-findings.md b/docs/detection-findings.md index f3eb30f..9898dfd 100644 --- a/docs/detection-findings.md +++ b/docs/detection-findings.md @@ -838,3 +838,110 @@ Peter's manual-skip logging idea is the obvious candidate, since it supplies lab detector nor the feed can. Had only LeVar been measured, the floor would have been lowered. + +## 2026-11-02: what the corpus says now, and the tier that is being acted on by accident + +The previous entry ended by naming what was missing: "a mechanism that can tell a real short ad +from a short coincidence". Words are that mechanism, and three of them were built and compared. +This entry records what the **live corpus** says, re-derived from `state.json` rather than quoted, +because the figure in `publish.go`'s own comment turned out to describe a subset. + +### The corpus, measured 2026-11-02 + +| | | +|---|---| +| feeds subscribed / shows tracked | 44 / 43 | +| episodes known from feeds | 9,020 | +| mirrored (audio on disk) | 336 | +| with words (whisper or published) | 327 | +| with a master to subtract against | 91 | +| episodes publishing audio-signature breaks | 267 | +| episodes publishing word breaks | 243 | +| word break spans published | 597 `cue`, 49 `ad` | +| listener labels recorded | 26 | + +### Subtraction's accuracy depends entirely on where the master came from + +`publish.go` says subtraction "measured 92% to 104% ... across eighteen episodes". That is still +true of the episodes it was measured on, and it is **not** true of the corpus. Across the 24 +episodes that now publish an `ad` span and have a length gap to check it against, recovery runs +from 20% to 146%, median 76%. + +Split by what the master actually is, the spread stops being a spread: + +| master source | episodes | median recovery | range | +|---|---|---|---| +| publisher transcript | 9 | **102%** | 94% – 104% | +| refetch (a second download of the same file) | 15 | **33%** | 20% – 146% | + +(Heavyweight's seven carry an empty `masterSource`: the master file predates the field, and +`Reclaim` faithfully restored the blank. They are publisher transcripts.) + +A publisher's transcript is a second *description* of the programme, written once, and +subtracting it works -- 94% to 104%, every pod found, edges pulled back off the speech. A refetch +is a second *copy of the same dynamically-assembled file*, carrying its own ads at its own +positions, so the diff is comparing two ad loads rather than finding one. It recovers a third of +the advertising, which matches the 21% and 0% measured on the refetch detector when it was first +tried and rejected on exactly those grounds. + +### The tier that was rejected is being published anyway + +Nothing in `Publishable` looks at `MasterSource`. It checks the candidate set against the length +gap and nothing else, so a refetch-derived candidate publishes on the same terms as a publisher- +derived one -- and refetch masters are now **15 of the 24** episodes publishing an `ad` break, +the majority of a tier the design had decided not to act on. + +The 33% is a real measurement, not an artefact: the refetch shows it rests on have large, +consistent length gaps (LeVar Burton Reads 3.39m median across 11 episodes, Darknet Diaries 3.31m, +Sources & Methods 3.72m, Serial 3.22m), so there is a solid yardstick to miss two-thirds of. + +### The length gap is only a guard where it is large and consistent + +The second finding, and the one nearly recorded wrongly. Two Reply All episodes publish `ad` spans +claiming 146% and 138% of their length gap, just under `overshootLimit`: + +``` +#188 Into the Depths declared 42.6m served 43.6m gap 1.01m + published: one ad span, 1154.2s -> 1243.0s (89s) = 146% of the gap +``` + +The obvious reading is that the detector over-claimed by 28 seconds and will skip programme. That +reading does not survive looking at the show rather than the episode. **Reply All's length gaps +across its eight mirrored episodes run from -0.48 to +1.02 minutes** -- they straddle zero. The +feed's declared durations are not a baseline for this show at all; the "61 seconds of inserted +advertising" that 146% is a percentage *of* is indistinguishable from encoder and feed noise. + +So the number being exceeded is itself noise, and which of the two is wrong on #188 cannot be +settled from this data. What can be settled is the structural point: `HasInsertedAdvertising` asks +only whether *this episode's* gap clears one ad unit, and 60.6 seconds clears 15. It has no notion +of whether the show's gaps are consistent enough for the figure to mean anything. Compare the +shows where it does mean something -- Incels 11.57-14.54m across 8 episodes, Heavyweight +0-10.06m, The Secret World of Roald Dahl 4.83-6.16m -- with the shows whose gaps sit in the +noise: Reply All -0.48 to 1.02, Jon Stewart -1.48 to 0.52, Sleepy History -1.57 to 0. +A per-episode threshold cannot tell those apart, and the per-show spread is sitting right there in +the state file, uncomputed. + +### Do not + +- **Do not quote `publish.go`'s 92-104% as the detector's accuracy.** It is the publisher-master + figure. Say which master. +- **Do not raise `overshootLimit` to admit more.** On the shows where the gap is meaningful it has + never been the binding constraint; on the shows where it binds, the gap is noise. +- **Do not read a single episode's gap as ground truth without checking the show's spread.** That + mistake was made and caught in the course of writing this entry: the 146% was about to be + recorded as 28 seconds of programme being skipped. +- **Do not rebuild the ad/furniture classifier.** Measured and deleted: 33% agreement with the + length gap, 263 advertisements called furniture, and the markers it keyed on appear in 5% of ads + and 4% of furniture -- it was reading noise. Recurring text publishes as `cue` instead, which + claims only that the audio repeats across the show's episodes, which is what was actually + measured. + +### The open questions, in order of how answerable they are + +1. **Should a refetch master publish `ad` at all?** The data says the tiers are not comparable + (102% vs 33%) and the code treats them as if they were. Answerable now; it changes what gets + skipped on fifteen episodes. +2. **Should the length-gap guard consider the show's spread rather than the episode's gap?** The + figures above are computable from `state.json` for every show. A show whose gaps straddle zero + has no usable yardstick and arguably should not publish a subtraction-derived `ad` either. +3. Whether #188's 89-second span is actually advertising. Needs a listen, not a measurement. diff --git a/server/internal/transcript/publish.go b/server/internal/transcript/publish.go index 670a8d5..b9856c3 100644 --- a/server/internal/transcript/publish.go +++ b/server/internal/transcript/publish.go @@ -10,17 +10,30 @@ decides what the phone will seek past, so it is deliberately narrow. Two detectors produce candidates and they are not close in quality. Subtraction -- comparing an episode's words against a second description of the same programme -- measured 92% to 104% of -what the declared-versus-served length gap implies across eighteen episodes, finding every pod, -with its edges pulled back off the programme. Reading the ad copy measured 0%, 0%, 15% and 29% -on the same kind of episode. Only the first is acted on. +what the declared-versus-served length gap implies, finding every pod, with its edges pulled back +off the programme. Reading the ad copy measured 0%, 0%, 15% and 29% on the same kind of episode. +Only the first is acted on. + +**That 92-104% is the figure for a publisher's transcript**, which is a second description of the +programme written once. Against a refetch -- a second download of the same dynamically-assembled +file, carrying its own advertising at its own positions -- the same code recovers a median of 33% +across fifteen episodes, with a range of 20% to 146%. Measured on the live corpus 2026-11-02; see +docs/detection-findings.md. Nothing below distinguishes the two, which is a known gap, not a +decision. The length gap is both the measure and the guard. It comes from the feed and the file rather than from any detector, which makes it the one number able to contradict one. */ // overshootLimit is how much more than the lengths imply a candidate set may claim and still be -// published, as a percentage. The measured spread runs to 104%; 150 leaves room for a music -// sting at the edge of a pod without leaving room for having found the wrong thing entirely. +// published, as a percentage. 150 leaves room for a music sting at the edge of a pod without +// leaving room for having found the wrong thing entirely. +// +// It was set against the publisher-transcript spread, which runs to 104%, so it sat well clear of +// the distribution. Where it actually binds is the shows whose length gap is noise rather than a +// measurement -- Reply All's gaps run from -0.48 to +1.02 minutes across eight episodes, so a +// percentage of one of them means little either way. Raising this limit would not help and +// lowering it would not either; see docs/detection-findings.md for what would. const overshootLimit = 150 // Publishable returns the candidates fit to skip, or nothing and the reason. |