summaryrefslogtreecommitdiff
path: root/docs/detection-findings.md
blob: 1bb78806a473209e3df60232031f9bac34d71180 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
# Tier 3 detection: what is known, and the root cause of why nothing confirmed

Written 2026-09-29, in four passes across the same day as the picture sharpened and then got
fixed. The correlation algorithm and its tuned constants were never the problem. Up First
confirmed nothing because `RetentionPolicy`'s one-day grace period usually reclaimed a played
episode's file before the *next* one was downloaded and processed, for a show this fast and this
promptly listened to -- so the detector kept discarding its only pending candidate and
restarting, rather than ever getting two episodes' files to compare at once. See "The live test,
and the actual root cause" below for the mechanism, and "The fix (made)" for what actually
shipped -- protecting a show's pending episode from reclaim, for both fingerprinting mechanisms.
**Not yet re-verified on a device.**

The rest of this document is kept in the order it was actually found, including the two
hypotheses that turned out to be wrong -- both cost real time to check and are worth not
re-checking blind next time.

**First pass:** the algorithm was suspected. It isn't -- see "What the desktop session actually
found." The production algorithm, run against real Up First audio at its real defaults, confirms
cleanly on every pair tested, both ends.

**Second pass, the phone reconnected the same day:** pulled the live database and WorkManager's
own execution history straight off it (read-only, via `run-as`). The pipeline is not broken --
**Cruising the Waves Podcast reached a real Tentative candidate (confirmedCount=2) that day.** Up
First specifically was stuck: its `show_fingerprints` row hadn't updated since its *first* episode
was ever processed, even though a second real episode was downloaded, played and decoded
successfully (by a different worker) the next day. `FingerprintUpdater` upserts on every outcome,
match or not, so that frozen timestamp meant the second episode's comparison never ran at all --
not a score that failed. See "What the phone actually showed" for the detail.

**Third pass, a live test on the phone:** triggered a fresh download and watched it happen in
real time. That's what found the actual mechanism -- see "The live test, and the actual root
cause."

**Fourth pass, the fix:** Peter picked exempting a pending episode from reclaim over the other
two options. Shipped for both intro/outro and midroll -- see "The fix (made)."

---

## What is settled

**The pipeline runs on hardware.** `MediaCodecClipSource` decodes, `FingerprintWorker` completes
SUCCESS per download, the re-decode path for the pending episode is exercised. This was never
true before 2026-09-28 and two real defects were hiding behind it (below).

**The confirmation threshold earns its place.** Three independently-fetched episodes must agree
before a candidate is trusted. The original prototype found why: two Darknet Diaries episodes
fetched minutes apart agreed on a segment that turned out to be a rotating inserted ad, not the
show's intro. Everything below makes that trap *more* likely, not less. Do not lower it to make
something confirm.

**Storage limits are real and were learned the hard way.** Android reads every Room row through
a CursorWindow that refuses anything over 2MB *on the way out* — the insert succeeds and the
next read throws. A 240-second clip at 8kHz 16-bit is 3.84MB. Intro/outro pending clips are no
longer stored at all (the episode id is, and the audio is re-decoded); midroll candidate clips
live in files. `StoredClip` owns the limit and `StoredClipSizeTest` pins it.

---

## The finding that changes the problem

**Dynamic ad insertion is not a detail; it is the environment.**

Radiolab declares Grass Apes at `00:22:05`. Measured on two separate fetches of the same
episode: **29:34 once, 27:39 another time.** The ads are stitched in per request, so:

- the served file's length is not a property of the episode,
- the ad *content* differs between two downloads of the same episode,
- and anything at a fixed offset in one file is at a different offset in the next.

`insertedAdMs` (declared vs measured) now exposes this per episode in the app. It is free, needs
no audio analysis, and is the obvious cross-check on the detector: **whatever the midroll scan
finds should add up to roughly that much.** If it finds 30 seconds where the gap says seven
minutes, the scan is missing breaks; if it finds more, it is matching content.

This reframes the intro/outro question. The head window is 240 seconds, which was chosen to
comfortably contain an intro. With a variable-length pre-roll in front of it, a show's real
intro sits at a *different offset in every file* — which the scan does handle, since it searches
the window rather than a fixed position — but the window now has to contain pre-roll **plus**
intro, and the pre-roll is the thing most likely to produce a false match between two
close-together fetches.

---

## What the desktop session actually found

Run against the four Up First episodes already cached from the original prototype
(`npr/ep1..4_head.wav`, `npr/ep1..4_tail.wav`, 240s at 8kHz each), reusing the same
`normalized_xcorr`/`best_match` primitive as `AudioCorrelation.kt` -- unmodified, so this is a
direct check of what the app itself computes, not an approximation of it.

**The leading hypothesis is disproven.** Simulating `IntroOutroDetector.findCandidate` exactly --
an 8-second window, quarter-window step, global search in the other clip -- at the app's real
`DEFAULT_REFERENCE_SECONDS = 8` and `MIN_SCORE = 0.6`:

| pair | intro r | intro position | outro r | outro position |
| --- | --- | --- | --- | --- |
| ep1→ep2 | 0.821 | a=44.0s b=44.0s | 0.999 | a=184.0s b=166.9s |
| ep1→ep3 | 0.807 | a=44.0s b=46.7s | 0.994 | a=226.0s b=184.0s |
| ep1→ep4 | 0.998 | a=90.0s b=93.7s | 0.995 | a=184.0s b=166.4s |
| ep2→ep3 | 0.805 | a=44.0s b=46.7s | 0.997 | a=180.0s b=184.3s |
| ep2→ep4 | 0.864 | a=44.0s b=49.3s | 0.997 | a=168.0s b=167.4s |
| ep3→ep4 | 0.793 | a=46.0s b=48.7s | 0.998 | a=172.0s b=167.2s |

Every pair clears 0.6 at both ends, several by a wide margin. The 8-second window is not too
wide for the sting -- it finds it cleanly regardless of exactly how much of the window is
"extra" content around it.

**Pre-roll varies a lot, and the algorithm doesn't care.** The intro lands anywhere from 44s to
94s into the head window across these four episodes -- up to 50 seconds of variability -- and
the global search finds it at every position tried. This is the "competing hypothesis" from the
first draft of this doc, and it is real (pre-roll length does vary) but it does not defeat
detection, because `findCandidate` was already built to search the whole window rather than
assume a fixed position.

**The resampler is not it either.** `MediaCodecClipSource.resample` is naive linear
interpolation, not a proper anti-aliasing filter, and the worry was that downsampling 44.1kHz
mp3 to 8kHz that way could be degrading correlation enough to matter. Decoded the same episode's
native 44.1kHz stereo, downmixed and resampled with the exact linear-interpolation formula from
`MediaCodecClipSource.kt`, and compared: **0.9991 correlation against a proper (ffmpeg/swresample)
decode of the same audio**, and re-running the full production simulation with the naively
resampled clip still confirms at r=0.821 -- indistinguishable from the ffmpeg path. Ruled out.

**A same-episode re-fetch, three days apart, is byte-identical for ~50 seconds and then diverges
completely.** The exact same episode URL, fetched fresh today against a copy saved 2026-09-26:
zero-lag correlation is 1.000 for every 5-second block from 0s to 45s, then falls to ~0 (pure
noise) for every block from 50s to 240s. Whatever comes after the intro is not stable across
fetches -- consistent with, and sharper evidence for, the "dynamic ad insertion is the
environment" finding above -- but the intro itself sits inside the stable prefix, so this does
not threaten intro detection specifically. It is a fresh reason to be careful trusting anything
found much later in a head/tail window, and matches why outro detection uses the *tail*, not a
position computed from the (unreliable) declared duration.

**The Darknet Diaries trap re-checked (experiment 6) without a new fetch.** The midroll
prototype work already re-fetched this exact pair: a ~51-second segment at position 0 matches
at r≈1.000 across `dd_179` and `dd_178` (2026), `dd_177` (2026) and `dd_008` from **2017** --
nine years apart, fetched today. A truly rotating, per-request DAI ad reproducing identically
against a specific nine-year-old archived fetch, by chance, is not plausible. This still looks
like Darknet Diaries' real, stable intro, not the trap the threshold exists to catch -- which
doesn't make the threshold wrong (a genuinely rotating ad would still be caught the same way),
just confirms this particular historical match was likely a false alarm rather than evidence of
one.

---

## What the phone actually showed (2026-09-29, later)

The phone reconnected mid-session. Pulled `listen.db` (via `run-as` + `adb exec-out cat`, no
device modification) and `no_backup/androidx.work.workdb` -- WorkManager's own execution
history, which nothing in this app writes to but which turned out to answer the question the app
state alone couldn't.

**The pipeline is not broken.** `show_fingerprints` for **Cruising the Waves Podcast** shows
`kind=INTRO, confirmedCount=2, referenceClipPcm` populated -- a real Tentative candidate, found
today, from an ordinary download. Something *does* reach Tentative on this exact build, on this
exact phone. The mechanism works in general; Up First specifically is the anomaly, not Tier 3.

**Up First's `show_fingerprints` row is frozen at its very first episode.** Both INTRO and OUTRO
show `confirmedCount=0`, `referenceClipPcm=NULL`, `pendingEpisodeId` pointing at one single
episode ("Powerful storms hit the US..."), and `updatedAtMs = 2026-09-27T02:19` -- the moment
that first episode was processed. A second real episode ("Can clergy candidates help Democrats
win in red states?") was downloaded, played and swept by retention the *next day*, and its
`midrollCheckedAtMs` is set, proving `MidrollWorker`'s `decodeFull` read that file successfully
that day. But `show_fingerprints.updatedAtMs` never moved again.

That is the specific, damning fact: `FingerprintUpdater.processEnd` calls `fingerprints.upsert(...)`
on **every** outcome -- a confirmed match, a failed match, all of it -- so if a comparison
involving the second episode had run at all, this row would show a later timestamp regardless of
whether it found anything. It doesn't. **The second episode's `FingerprintWorker` run either was
never enqueued, or `decodeHead`/`decodeTail` both returned null for it** -- and the latter looks
unlikely given `decodeFull` (the same underlying decode, just a different window) succeeded on
the identical file later that day. `EpisodeDownloader.downloadOverHttp` was read end to end look
for a race between "downloaded" and "playable" -- there isn't one, it's a fully blocking
download-to-`.part`-then-atomic-rename before `onDownloaded` fires, so a half-written file is not
the likely explanation either.

WorkManager prunes completed `WorkSpec` rows after roughly a week, so the actual historical
`FingerprintWorker` run (if any) for that second episode is already gone -- only today's runs and
the next periodic `MidrollWorker` firing were still in the table. This is now a live-observation
question, not an archaeology one.

## The live test, and the actual root cause

Triggered a fresh Up First download on the phone and watched `androidx.work.workdb` and
`show_fingerprints` immediately after, live:

- A new `FingerprintWorker` `WorkSpec` row appeared within seconds and reached `SUCCEEDED`.
- `show_fingerprints` for Up First **did** update -- `updatedAtMs` moved to right now, for the
  first time since 2026-09-27. But `pendingEpisodeId` didn't advance to a *second* episode to
  compare against; it just replaced the old one with this new download. `confirmedCount` stayed
  0, `referenceClipPcm` stayed null.

That is `checkAgainstPending`'s own designed behaviour when `pendingClipOf` returns null --
which it does whenever the *stored* pending episode's file no longer exists to re-decode. And it
doesn't: the previous pending episode ("Powerful storms...") was played on 2026-09-27, and
`RetentionPolicy`'s grace period is **one day** (`gracePeriodMs = TimeUnit.DAYS.toMillis(1)`,
`Repositories.kt`). By the time any later Up First episode is downloaded and processed, retention
has very often already reclaimed the previous one's file -- so instead of ever comparing two
episodes, the detector keeps discarding its only candidate and starting over from whatever the
newest download is.

**This is the real mechanism, not a bug in the correlation math.** Up First is a fast, daily,
~15-30 minute news show that gets played through quickly -- close to the worst case for this
race. A show downloaded weekly and left to sit would have days of margin between a play and the
next download; Up First has almost none. Cruising the Waves reaching Tentative today is
consistent with this: whatever gap that show's two most recent downloads happened to land at was
wide enough, or its previous pending episode hadn't been played+reclaimed yet.

**What's still not fully explained:** the *second* episode ever downloaded for Up First
("Can clergy candidates...", 2026-09-28) left no trace in `show_fingerprints` at all -- not even
the "adopt as new pending" fallback just reproduced live, which would still have moved
`updatedAtMs`. Either its `FingerprintWorker` run was deferred long enough that it never actually
executed (Doze/battery-restriction deferral of a one-shot job, still unresolved days later), or
something force-stopped the app in a way that drops pending `WorkSpec` rows before they run.
Secondary to the retention-race finding above, and not reproduced in the live test.

## The fix (made)

Peter chose option 1: exempt an episode retention would otherwise reclaim while it is a show's
pending comparison basis, for both mechanisms. `RetentionSweeper` gained a `pendingEpisodeIds`
seam (`suspend () -> Set<String>`, defaulting to empty so every existing caller and test was
unaffected) and filters its reclaim candidates against it before the grace-period policy ever
sees them. `RetentionWorker` wires the real thing: the union of
`ShowFingerprintDao.getAllPendingEpisodeIds()` (intro/outro) and
`MidrollDao.getAllMidrollPendingEpisodeIds()` (midroll, reading
`SubscriptionEntity.midrollPendingEpisodeId`) -- both new, narrow queries added for exactly this.

Applied to both mechanisms, not just the one that got diagnosed: `MidrollUpdater`'s discovery has
the identical shape (a rolling pending episode compared against whatever downloads next) and
would race retention the same way for a fast show, just silently -- discovery finding nothing
that round rather than a visible frozen row, since it has no fallback state to get stuck in.

Not yet re-verified on a device: the next live test is downloading two Up First episodes close
enough together that the first is still within its (now-irrelevant, since it's protected) grace
period, and confirming `pendingEpisodeId` actually advances to a real second-episode comparison
instead of restarting again.

## Experiments, in order (updated)

1. ~~Quantify pre-roll variability.~~ **Done above** -- 44s to 94s across four episodes, and it
   does not defeat the search.
2. ~~Locate and measure a real intro.~~ **Done above** -- the table gives exact positions; the
   6-second figure from the earlier note was in the right neighbourhood but pre-roll length, not
   sting length, turned out to be the variable that mattered.
3. Sweep `MIN_SCORE` in a *falsely-confirms* direction, not a make-it-pass direction: now that
   the real default already confirms cleanly, the useful question is how much margin there is
   before it would start confirming something wrong, not whether a lower threshold helps.
4. **Check 240 seconds is still enough**, on a worse show than Up First. Still open.
5. **Validate midroll totals against `insertedAdMs`.** Still open; no `insertedAdMs`-bearing show
   was fetched fresh this session.
6. ~~Re-check the Darknet Diaries trap.~~ **Done above**, using data the midroll prototype
   session already had cached -- still looks like a real intro, not a trap.
7. ~~Get on-device state.~~ **Done above, and it's the one that mattered** -- see "The live test,
   and the actual root cause." Desktop simulation had gone as far as it could without it.

---

## Do not

- **Do not change `DEFAULT_REFERENCE_SECONDS`, `MIN_SCORE` or `CONFIRM_THRESHOLD` from the app.**
  They are tuned numbers and the app cannot show you what a change costs in false positives.
- **Do not persist a decoded clip in a database row.** See the CursorWindow note.
- **Do not read `EpisodeEntity.durationMs` when you mean the real length.** It is the feed's
  claim, it is rewritten on every refresh, and on these shows it is minutes short. Use
  `playbackDurationMs`.
- **Do not lower the confirmation threshold to make something confirm.** Failing to skip is a
  non-event; skipping real content is the failure that loses trust in the feature.

---

## Environment note

YouTube extraction and resolution **cannot be exercised from tungsten** — every watch URL gets a
302 to a bot-check page. Anything touching YouTube has to be verified on the phone. Podcast
feeds, Room migrations, playback and the fingerprint pipeline are all fine on the emulator.

---

## Fifth pass, 2026-10-03: the cost nobody had measured

**Peter's report:** the phone runs hot and uses a lot of battery, and intelligent skipping has
never done anything to any episode.

Everything above is still true and the retention fix still stands. It was not the whole story,
and on the evidence below it was not the dominant blocker either.

### The candidate search cost 199 seconds per end, measured

`IntroOutroDetector.findCandidate` bootstrapped a show by sliding an 8-second window across a
240-second clip in quarter-window steps -- **117 positions** -- and running a *full-length* FFT
cross-correlation against the whole of the other clip at every one of them, re-transforming that
unchanged clip each time. 702 transforms of 2^21 points per episode.

Measured, not estimated, by `IntroOutroDetectorScaleTest` at the real sizes and real constants:

| path | before | after |
| --- | --- | --- |
| candidate search, one end | **199.1 s** | 17.8 s |
| ditto, full resolution with the target transform hoisted | 136.6 s | -- |
| confirming against an existing reference | 1.7 s | 1.7 s |

That is on a desktop. A phone has a fraction of this machine's memory bandwidth.

### Why that meant nothing was ever persisted

`FingerprintWorker` is a plain WorkManager worker, so **Android stops it at about ten minutes**.
Both ends of an episode run inside one worker, and `FingerprintUpdater` writes nothing until a
comparison returns -- so the worker was killed mid-correlation, saved nothing, and WorkManager
re-enqueued it to start the same work again with backoff.

The expensive path is specifically the *bootstrap*, the one that runs until a show has a
reference clip. Escaping it requires one run to finish. None ever did. This is a complete
explanation of both of Peter's reports at once: hours of full-CPU work, and a
`show_fingerprints` row frozen at the first episode ever processed.

It also fits the second pass's observation better than the retention theory does. That pass found
Up First's row frozen even though a second episode "was downloaded, played and decoded
successfully (by a different worker)", and concluded the comparison "never ran at all". On this
evidence the comparison *did* run, for ten minutes, and died before its `upsert`. Both mechanisms
produce a frozen row; only one of them also produces a hot phone.

**One data point still does not fit:** Cruising the Waves reached `confirmedCount=2` that day, so
something completed once. Worth checking whether its episodes are short enough to have come in
under the limit.

### The fix

Coarse-to-fine, the composition `MidrollUpdater` already used and this document already
defended: decimate both clips by 8, rank candidate positions on the decimated copies, then
re-slice the best three at native resolution and let a full-resolution correlation decide.
Plus `AudioCorrelation.prepareTarget`, so the target clip is transformed once per search rather
than 117 times.

**10.6x faster, and provably the same answer.** `the coarse search picks the same segment full
resolution does` asserts the chosen reference window is identical sample for sample -- both
searches step the same native grid, so the positions are directly comparable. No detection
threshold was touched.

### Midroll detection could never have run at all

Separately and independently: `MediaCodecClipSource.decodeFull` buffered every interleaved sample
at the file's native rate, then every mono frame, and only then resampled to 8kHz.

| episode | native PCM buffer | mono buffer | both live at once |
| --- | --- | --- | --- |
| 10m | 106 MB | 212 MB | 318 MB |
| 13m (Up First) | 145 MB | 289 MB | 434 MB |
| 30m | 308 MB | 616 MB | 924 MB |
| 60m | 611 MB | 1,221 MB | 1,832 MB |

There is no `largeHeap` in the manifest. It threw `OutOfMemoryError`, the `runCatching` in
`decode` swallowed it, `decodeFull` returned null, and `MidrollUpdater` then marked the episode
**checked forever**. So no episode of any normal length was ever scanned, none would be retried,
and the readout said "None found" rather than anything true.

Two changes: `MonoResampler` consumes the decoder buffer by buffer and keeps only the 8kHz
output, so the peak allocation is the result rather than the input; and `MidrollUpdater.Tuning`
now has a `maxDecodableMs` (20 minutes) that skips a longer episode **before** spending anything
and **without** marking it checked.

**Still outstanding:** an episode over that ceiling is not scanned. Lifting it properly means
decoding candidate ranges on demand (`decodeRange`) instead of holding the whole body, which is
a real change to `MidrollUpdater.discover`/`extractSegment` and to `MidrollDetector.check` --
all three currently want the entire native-resolution array. Left deliberately undone rather
than half-done.

---

## The playback-start pause: one hypothesis disproven, instrumentation shipped

Peter's third report: "there's a longish pause when starting playback, proportional to episode
length." Recorded here with the others because the next session should not re-check what has
already been eliminated.

**Nothing in the app's own play path scales with episode length.** Traced end to end: tap ->
`PlayerViewModel.play` -> `PlaybackAdvancer.start` (a few indexed row reads) ->
`PlayerConnection.play` -> `setMediaItem`/`prepare`. No file scan, no hashing, no probe.

**`PrepareTrace` now measures it**, tagged `ListenPlayback`, one line per start:

```
prepare ep=9c9750ef len=1642s src=local ready=666ms playing=667ms
```

**Measured on the emulator, a 27-minute downloaded Radiolab episode: ~400-500ms to READY**, with
no other load on the device. So there is a real cost here and it is not only CPU starvation from
the fingerprint worker.

### Disproven: the missing MP3 seek table

The downloaded file has **no Xing, Info or VBRI header** (checked directly: ID3 tag of 166 bytes,
audio from byte 176, none of the three markers present). Without one, Media3 has no seek table to
read, and the obvious suspicion was that `Mp3Extractor` was reading through the file to build one
-- which would be length-proportional, exactly matching the report.

Tested by enabling `Mp3Extractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING` on a
`DefaultExtractorsFactory` and measuring the same episode:

| configuration | ready (ms) | mean |
| --- | --- | --- |
| shipped default | 404, 531, 386 | ~440 |
| constant-bitrate seeking on | 469, 450, 404 | ~441 |

**No difference.** The distributions overlap entirely. A first pair of samples had looked like
666ms against 411ms, which is a 38% improvement and was cold-start noise -- the lesson being that
one sample of each proves nothing here, because the spread is wider than any effect.

The flag was reverted. It would also have been the wrong trade even if it had worked: constant-
bitrate seeking makes seeks approximate on a genuinely VBR file, and playback position is the one
piece of data in this app that cannot be reconstructed.

### What is still open

Whether the ~440ms scales with episode length. It cannot be settled on the emulator, which has
one episode on it. The instrumented build is on the phone, so **two plays of episodes with very
different lengths will answer it from the log** -- compare `len=` against `ready=`. If it is flat,
the pause is a fixed preparation cost and the "proportional" impression was the fingerprint
worker saturating the CPU, which is now fixed. If it scales, the cause is inside Media3's
preparation and the next place to look is what `ProgressiveMediaPeriod` reads before reporting
READY.

---

## 2026-10-05: detection ported to Go and verified on real audio

Peter's call: detection moves to a server, in Go, matching podkit and doot. The port is in
`server/`, and this section is what it actually proves -- which is not the same as what it is.

### Verified

**Against numpy, not against itself.** `AudioCorrelationTest`'s numpy-derived fixtures were
extracted mechanically (not retyped) into `server/internal/detect/fixtures_test.go`, with the
same pinned literals: self-match 1.0, embedded sting r=0.9998959789695058 at position 120,
unrelated target r=0.3672490793439344. The Go FFT is separately checked against a direct DFT
computed from the definition, and the whole FFT correlation path against a brute-force Pearson
correlation. That is three independent anchors, because this is the second port of this maths
and the first one shipped a normalization bug.

**On real audio, end to end.** Four Radiolab episodes, fetched from the live feed:

```
intro: confirmed on 4 of 4 episodes; self-check passed on 6 pairs, worst r=0.9286
  ep1  21:07   skip intro to 0:56.000
  ep2  22:05   skip intro to 0:16.652
  ep3  31:02   skip intro to 0:16.652
  ep4  1:03:27  skip intro to 1:13.152
detection took 2m9s
```

The intro sits at a different offset in each episode -- 48.0s, 8.65s, 8.65s, 65.15s -- which is
the varying pre-roll this document has described since the first pass, now observed directly in
production code. Cross-checking the located segments against each other at exact sample
positions gives 0.93 to 0.998 on all six pairs: it is genuinely the same recording in all four.

### What the self-check caught, immediately

`skips.SelfCheck` correlates the *located segments against one another*, rather than each
against the reference they were found with. Those are different claims. Correlation is not
transitive: two segments can each clear 0.6 against a reference while scoring near zero against
each other, and offsets derived from such a set would cut different audio out of different
episodes.

On the very first real run it rejected the outro:

```
outro: confirmed on 3 of 4 episodes; SELF-CHECK FAILED: ep1 vs ep2 at r=0.7460 -- offsets withheld
```

Before the check existed, that same run published outro offsets of 0:12, 0:11 and 0:11 -- which
would have ended three episodes early at a point two of them do not share. **The confirmation
threshold passed a false positive and the self-check stopped it.** Offsets are now only
published when the segments verify against each other, and `listen-detect` exits non-zero when
they do not.

Outro detection on this show is therefore still unproven. It fails closed, which is the
important part, but why a 0.746 pair cleared a 0.6 threshold is the next thing to look at.

### A false alarm worth not repeating

The first hand check of the real run showed r=0.03 between two located segments and looked like
a serious bug. It was an artifact of reading offsets the CLI had rounded to whole seconds: the
true position was 8.65s, so the extracted window was 0.65s out, while two episodes that happened
to share the same rounded offset lined up perfectly and scored 0.997. The CLI now prints
milliseconds, and `SelfCheck` tolerates half a second of position error for the same reason.

### Not verified, and not built

- **No server.** What exists is the detection core and a CLI. The feed poller, the episode
  mirror, the 8-episode retention, the HTTP API and the phone changes are all still to come.
- **Dynamic ad insertion did not reproduce from this host.** All four episodes served *exactly*
  their declared length -- Grass Apes at 22:05, against the 29:34 and 27:39 measured earlier
  from the phone. So either insertion is conditioned on the client (user agent, geography,
  headers) or it is not always on. This matters in both directions: it is the entire reason the
  server has to serve the audio it detected against, and if the server reliably gets the clean
  file then mirroring strips the advertising rather than merely locating it. Four samples from
  one host with curl is not enough to conclude either way, and it should be measured properly
  before the design leans on it.
- **Midroll detection is not ported.** Only intro/outro.