summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/detection-findings.md74
1 files changed, 74 insertions, 0 deletions
diff --git a/docs/detection-findings.md b/docs/detection-findings.md
index 1bb7880..26b31b3 100644
--- a/docs/detection-findings.md
+++ b/docs/detection-findings.md
@@ -495,3 +495,77 @@ milliseconds, and `SelfCheck` tolerates half a second of position error for the
one host with curl is not enough to conclude either way, and it should be measured properly
before the design leans on it.
- **Midroll detection is not ported.** Only intro/outro.
+
+---
+
+## 2026-10-05, later: the server ships, and simultaneous mirroring defeats the threshold
+
+`server/` now runs on tungsten as `listend.service`, behind Apache at https://listen.terst.org
+with a Let's Encrypt certificate. It polls the phone's subscriptions, mirrors the newest 8
+episodes per show, detects, and serves both the offsets and the audio they belong to. The phone
+pushes its feed list from `SubscriptionRepository.refreshAll`, prefers the server's audio when it
+has it, and writes the offsets into the same `detectedIntroMs`/`detectedOutroMs` columns the
+on-device detector used -- so playback needed no changes at all. `FingerprintWorker` and
+`MidrollWorker` are deleted.
+
+Verified live, phone to server: `POST /api/subscriptions` returned 200 from the app, the server
+woke, mirrored 8 Radiolab episodes, and detected across all of them in 3m59s.
+
+### Dynamic ad insertion is on every episode, and it is not subtle
+
+Every one of the 8 came back longer than declared:
+
+| episode | declared | served | inserted |
+| --- | --- | --- | --- |
+| Grass Apes | 22:05 | 28:18 | 6:13 |
+| The Sweetest Thing | 31:02 | 37:15 | 6:13 |
+| 23 Weeks 6 Days | 63:27 | 72:26 | 8:59 |
+| Nigel and the Real Gannet | 25:54 | 33:01 | 7:07 |
+| The Shoe Thrower | 77:29 | 87:02 | 9:33 |
+
+This also corrects the earlier pass in this document, which found declared == served on four
+fetches made with plain `curl` and doubted whether insertion was happening at all. It is, on
+every episode, when fetched by the server. Why `curl` got clean files and the server does not is
+unexplained and worth knowing -- the only obvious difference is the User-Agent.
+
+### The confirmation threshold does not survive simultaneous fetching
+
+Detection reported the intro confirmed on 8 of 8, self-check passed. **It had locked onto an
+advertisement.**
+
+Proved, rather than suspected: the 8-second segment the detector chose was extracted from the
+server's copy of Grass Apes and searched for in the `curl` copy of the same episode fetched
+hours earlier with no inserted advertising. **r = 0.0621.** It is not in the programme. It is in
+the ad break.
+
+The mechanism is the Darknet Diaries trap at a new scale. The threshold's whole premise is that
+three *independently fetched* episodes are unlikely to share an inserted ad. Mirroring 8
+episodes within the same minute from the same ad server makes them maximally dependent: whatever
+campaign is running is in all 8, identical, and the detector finds it with scores of 0.93 to
+1.00. The self-check cannot catch this -- it only proves the located segments are the same
+audio, and they genuinely are.
+
+This is *worse* than the phone's original design, where episodes arrived days apart and so were
+genuinely independent. Moving to a server fixed the cost problem and broke the independence
+assumption the safety mechanism rests on.
+
+### What the published offsets currently are
+
+Head-window matches only, so every offset lands inside the first four minutes, and skipping to
+the end of a matched segment skips pre-roll advertising. That is useful by accident, not by
+design, and it will move unpredictably whenever the ad campaign rotates. **It is not intro
+detection and must not be described as such.**
+
+### The fix, not yet made
+
+Record when each episode's audio was fetched, and require the agreeing episodes to span more
+than one fetch occasion before a segment is trusted as the show's own. The mirror already rolls
+forward one episode at a time as feeds update, so independence arrives naturally -- it is only
+the initial batch that is simultaneous. Expect no offsets at all for the first day or two after
+a show is added, which is the correct answer rather than a confident wrong one.
+
+There is a reframing worth considering instead, which the data argues for: what this reliably
+detects, with very high confidence, is **the inserted advertising itself** -- because
+simultaneous fetches share it exactly. Skipping ads is the actual goal, and the declared-versus-
+served gap is a free, independent check on whether a scan has accounted for all of it. Detecting
+ads directly may be a better target than detecting intros and hoping ads do not get in the way.