diff options
| author | Claude <noreply@anthropic.com> | 2026-10-19 09:15:03 +0000 |
|---|---|---|
| committer | Claude <noreply@anthropic.com> | 2026-10-19 09:15:03 +0000 |
| commit | 10abc31e8220b39231cac31ee96109071c56c950 (patch) | |
| tree | f914eb6ddf47b974547d85376c6be185627a3923 /docs | |
| parent | d96c876752e95a17169d18729dd1897d2dc5731f (diff) | |
Record that lowering the noise floor to 8s is wrong
The 12s floor cut real detection on shows with verified ads -- LeVar
Burton Reads fell from 19.0 to 10.5 minutes found against a 26.6 minute
gap -- so 8s looked like the fix. Measured on an isolated copy of the
live data it is not: Sleepy History at 8s finds 174 minutes of
"advertising" in a show that has 48 seconds of it, worse than the 4s
floor that produced Peter's original complaint.
MinRunSamples and MergeGapMs interact and cannot be tuned separately. A
low floor lets noise fragments survive and the 60s bridging then welds
them into enormous spans.
Had only LeVar been measured the floor would have been lowered.
Also re-pins .coverage-baseline, which the hook left modified after
d96c876 deleted covered readout code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/detection-findings.md | 39 |
1 files changed, 39 insertions, 0 deletions
diff --git a/docs/detection-findings.md b/docs/detection-findings.md index 6f48b72..f3eb30f 100644 --- a/docs/detection-findings.md +++ b/docs/detection-findings.md @@ -799,3 +799,42 @@ breaks" on 111 of them, 74 minutes in total. Those are not advertisements. They own recurring furniture -- intro, break cue, outro -- which is worth skipping but is a different thing, and reporting it as inserted advertising is wrong. Every mirrored episode has a usable declared duration, so this classification is always available. + +## Lowering the noise floor to 8s: measured, and badly wrong + +The 12-second floor was set on 2026-10-15 after Peter reported "madness" — 37 breaks in one +episode, 14 of them exactly 4.0s, the old floor. It worked: his episode went 37 breaks to 1, and +every sub-12s fragment in the corpus disappeared. + +But it also cut real detection on shows with verified advertising. LeVar Burton Reads went from +19.0 minutes found to 10.5, against the 26.6 the length gap implies. That looked like the floor +being too aggressive, and an 8-second floor looked like the fix. + +**It is not.** Measured 2026-10-19 on an isolated copy of the live data, same episodes, same +batch, production code with only `MinRunSamples` changed: + +| show | 12s floor (shipped) | 8s floor | length gap implies | +|---|---|---|---| +| LeVar Burton Reads | 10:30 | **19:43** | 27:35 | +| Sleepy History | 14:11 | **174:44** | 0:48 | + +Sleepy History at 8 seconds finds **174 minutes** of "advertising" in a show that has 48 seconds +of it — worse than the 48:03 the original 4-second floor produced, and three hours of a listener's +programme proposed for deletion. + +**The two parameters interact, and that is the whole lesson.** `MergeGapMs` was widened to 60s in +the same change that raised the floor. A high floor plus wide bridging is fine: few fragments +survive, and bridging joins the ones that genuinely belong together. A low floor plus wide bridging +is catastrophic: scattered noise fragments survive *and* get welded into enormous spans. Neither +parameter can be tuned by looking at it alone, and the 8s number was chosen by post-hoc filtering +of already-detected breaks, which cannot model a detector-level change — the same modelling mistake +that produced the wrong "31% to 34%" prediction for the floor raise itself. + +**The floor stays at 12 seconds.** The cost is real and should be stated plainly: on shows with +verified ad insertion the scan now finds roughly 38% of what the feed implies, down from about 69% +at the old floor. That is the price of not proposing to skip three hours of a sleep podcast. Buying +the detection back needs a mechanism that can tell a real short ad from a short coincidence — +Peter's manual-skip logging idea is the obvious candidate, since it supplies labels neither the +detector nor the feed can. + +Had only LeVar been measured, the floor would have been lowered. |
