Let the listener validate or invalidate a detected break from a popup

Keeps partners on episode_midrolls (schema 15), stores verdicts in their own
break_verdicts table so a server refresh cannot wipe them, and drops invalidated
breaks from skipping on the next tick. Verdicts are local only.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

1 files changed, 263 insertions, 0 deletions

diff --git a/docs/superpowers/plans/2026-10-20-corrections-and-signature-library.md b/docs/superpowers/plans/2026-10-20-corrections-and-signature-library.md
new file mode 100644
index 0000000..6dea531
--- /dev/null
+++ b/docs/superpowers/plans/2026-10-20-corrections-and-signature-library.md
@@ -0,0 +1,263 @@
+# Human corrections, then a signature library — plan and handover
+
+Written 2026-10-20, immediately before a context compaction. Assume the next session knows
+nothing. `CLAUDE.md` holds the standing rules; this holds where the work is and why.
+
+**Spec this argues from:** `docs/superpowers/specs/2026-10-08-furniture-and-cues-design.md`
+(furniture/ad distinction, partly built) and the design published at
+https://claude.ai/artifact/BJmBFVrohrJo1hB6vYSz2Y (signature library, nothing built).
+
+---
+
+## Where things stand
+
+**Branch `main`, clean at `f087b8d`.** 771 unit tests. Room schema **14**. APK published at
+https://static.terst.org/files/listen-debug.apk built from `f087b8d`.
+
+**Server healthy**, `listend.service` on tungsten, 43 shows, 335 mirrored episodes, 501 breaks.
+Up since 2026-10-17, zero restarts.
+
+### Shipped since the break-kinds plan
+
+| commit | what |
+|---|---|
+| `6c4d40e` | Refresh skips on feed refresh; `audioFromServer` provenance so server timestamps never reach publisher audio |
+| `203e8a6` | That sweep bounded to 6 concurrent instead of several hundred sequential |
+| `bd2a8ba` | Downloads **wait** for the server's copy instead of silently taking the publisher's; `WAITING_FOR_SERVER`; explicit publisher action |
+| `999759f` | A rejected token (401/403) is permanent, not a reason to wait; notice in Settings |
+| `d96c876` | Skip readout becomes three numbers plus tappable marks; the 37-row timestamp list deleted |
+| `2329c92` | **Episode screen becomes the player**; `PlayerScreen.kt` deleted; six buttons to one plus overflow |
+| `f087b8d` | Show name in the top bar, episode in the body; chapters fetched lazily |
+
+### In flight when this was written
+
+One subagent building the **tap-a-mark popup** (see next section). It will land a commit and a
+schema 14 → 15 migration adding `partners`, plus a verdict table. If the tree is dirty or an
+unexpected commit is present, that is it.
+
+---
+
+## The sequence, and why it is in this order
+
+Peter's instruction, verbatim: *"let me tap the highlighted skips - give me a popup that has meta
+data and a way to explicitly validate or invalidate it"* — offered as the answer to the risk that
+a wrong signature-library entry would cut the same audio from every episode forever, silently.
+
+**That reordered the plan.** Corrections come before the library, not after it, because the labels
+are the most trustworthy signal in the system — a human confirmed them — and they are the only
+source of **negative** evidence. Nothing else can say "this is not an advertisement".
+
+### 1. The popup — IN FLIGHT
+
+Tapping a break mark opens metadata and a verdict. Detail in the dispatch; the parts that matter:
+
+- **Persist `partners`.** The server sends it on every break and the phone discards it. It is the
+  "why" the popup exists to show. Schema 14 → 15.
+- **Verdicts live in their own table**, keyed by `(episodeId, startMs, endMs)` with a tolerance.
+  They must survive `ServerSkips.recordBreaks`, which **deletes and re-inserts every range for an
+  episode** on each refresh — a verdict stored on `episode_midrolls` would be wiped.
+- **Invalidate takes effect on the next tick**, no refresh, no round trip.
+- **Verdicts stay local for now.** No endpoint, no consumer. Storing them from today means a
+  corpus exists by the time the library does.
+
+### 1b. "This is an ad" in the notification — NOT STARTED
+
+Peter, 2026-10-20: *"The notification currently has duplicate skip buttons (our custom ones plus
+forward/back) - there's UI space for explicit 'this is an ad' there."*
+
+**He is right about the duplication.** `ListenSessionCallback.onConnect` leaves
+`COMMAND_SEEK_BACK` and `COMMAND_SEEK_FORWARD` in the player commands (they are in
+`DEFAULT_PLAYER_COMMANDS`), which Media3's notification provider renders itself, and then
+`NotificationControls.customLayout()` publishes a second seek-back-15 / forward-25 pair on top.
+Reclaiming a slot means dropping one pair.
+
+**Do this on a device, carefully.** `ListenSessionCallback` carries a comment recording that
+removing commands to free notification slots once silently killed the headphone skip button.
+Whichever pair goes, verify: the notification, the lock screen, Bluetooth/headphone transport,
+and Android Auto. Note also that that comment currently contradicts its own code — it says "Only
+previous-media-item goes" while the code removes both previous and next. Worth resolving while
+in there.
+
+**Why this button matters more than the popup.** They catch opposite errors:
+
+| surface | catches | evidence |
+|---|---|---|
+| tap a mark → popup | **false positives** — a mark that should not be there | negative label |
+| notification button | **false negatives** — an advertisement nothing detected | positive label |
+
+False negatives are by far the bigger problem: the scan finds about **38%** of the advertising the
+feeds imply, so roughly three fifths of what he hears is undetected. The popup can only ever
+correct what was already found. And the notification is available at the one moment he actually
+notices — phone in pocket, advertisement playing, without unlocking anything.
+
+**What one press captures, and what to do with it.** A press is a *timestamp*, not a range, and a
+signature needs a range. Do not ask him to press twice. Record the position and let the server
+work out the extent: take a window back from the press (he presses partway in, and spots run
+roughly 15–60s, so something like `[t-45s, t+15s]`) and feed it to the existing
+`detect.MatchedRuns` machinery as a *seeded* search rather than a blind one. That is the same
+correlation already in use, pointed at a known-positive location instead of groping.
+
+**The press also skips, and what happens next is more signal.** Peter, 2026-10-20: *"Also skip -
+if I hit back immediately that might invalidate the signal or indicate I jumped past the end."*
+
+That turns one button into a complete labelling gesture. The rule, which needs no extra UI:
+
+| gesture | means | what it fixes |
+|---|---|---|
+| press | advertising starts at or before here | **start** of the range |
+| press again, still in the jump | still advertising | **extends** the range |
+| back, landing inside the jump | overshot | **end** of the range, at the landing point |
+| back, landing before the first press | that was not advertising | **invalidates** the whole thing |
+
+The two back cases are told apart by *where he lands*, not by guessing: inside the jumped span is
+a correction to the end, before the original press is a rejection. Both are unambiguous from the
+playhead alone.
+
+**How far to jump is an ergonomics question, and probably not a constant.**
+
+An earlier draft of this plan argued the jump must be a fixed size and deliberately unequal to the
+25s forward-seek, so that a back-press would be "unambiguous". Peter challenged it and it does not
+hold, in either half:
+
+- The system knows **which button was pressed**. A following back-press is read as a correction
+  because the last action was the ad button, not because the distance was distinctive. Equal
+  distances would create no ambiguity anywhere in the logic.
+- The listener is not computing anything either. He presses back until the programme resumes, and
+  the correction is read from **where he lands** against a span the system recorded exactly. That
+  works for any jump size; he never needs to know the number.
+
+What actually sets the size is the trade between **too small** (several presses to clear one pod)
+and **too large** (routinely landing in content, so he corrects constantly). It should sit near the
+typical *remaining* advertisement duration from a mid-advertisement press.
+
+And it need not be fixed: if a detected break is nearby, jump to its end — that position is already
+known. If the episode's declared-versus-served gap says minutes remain, that is informative too.
+The press means "get me past this advertisement", so the best jump is the best estimate available
+at that moment. Correction semantics are unaffected, because "landed inside the jumped span" holds
+for any span.
+
+The one real reason to keep the two buttons distinct is that *he* must not confuse them in his
+pocket, which is a matter of icon and label, not distance.
+
+**"Immediately" needs a window.** A back-press within roughly 15–20 seconds of wall clock counts
+as correction; later than that it is ordinary navigation and must not be read as a label. Store
+the window with the label so a future change to it does not retroactively reinterpret old data.
+
+This composes with his earlier manual-skip-logging idea: *"a series of skips with small gaps
+between them, sometimes followed by one or more skips back, is a pretty strong signal of an ad
+block."* A press and a skip-burst are two observations of the same event, and a skip *back* after
+over-shooting brackets the end precisely. Treat them as one input stream.
+
+### 2. Push verdicts to the server — NOT STARTED
+
+Phone to server. Needs an endpoint and server-side storage. Design note: a verdict is about a
+*range of a specific episode's audio*, so it is only meaningful for episodes whose bytes the
+server itself served — gate on `audioFromServer`, the same invariant the refresh sweep uses.
+
+### 3. The index spike — NOT STARTED, and it is the gate
+
+**Do this before anything else in the library, because it decides whether the library is
+possible.** A linear scan of N signatures against each episode does not survive: at ~0.33s per
+correlation, 500 entries × 8 episodes is over twenty minutes per show against a pass that
+currently takes seven. That is the same cliff the original intro search went over (199s per end,
+killed by WorkManager before persisting anything).
+
+Needs landmark hashing — spectral peak pairs, hashed with their time offset, inverted index. A
+true match produces many hashes agreeing on one offset; noise scatters.
+
+**The spike has a ready-made pass/fail test**, from the probe run 2026-10-20:
+
+- It must find the Up First repeat that scored **r = 0.877** (template: episode
+  `32b5b954c1e705c185f51829.mp3` at 847s for 60s; found in `99e85f679bcf64db8739cfda.mp3` at
+  836s, and at r = 0.678 in `73ae08f072189cc954ea6b71.mp3`).
+- It must reject the non-matches that scored **0.06–0.13** (Tangle `7ad21fd746fe0d22440c9f6a.mp3`,
+  Serial `94f4ce3a94f5af2bf1b488aa.mp3`, Heavyweight `c7801ebf5c52825fbab680e8.mp3`).
+
+If it cannot do both, fast, the library stops there and nothing else is wasted.
+
+### 4. Store and age signatures — NOT STARTED
+
+Two kinds, which behave nothing alike:
+
+- **Cue** — a show's own furniture (break stinger, opening, credits). One show, permanent while
+  subscribed. Search Engine's is 8.0s and recurs 3–4 times an episode; Jon Stewart's ~5s at four
+  positions.
+- **Copy** — one advertisement. Any show, aged out after months unseen.
+
+Confidence is a count of sightings across **separate fetch occasions**, which is also the
+independence test the withheld intro/outro offsets have been waiting for. Never promote on one
+sighting.
+
+### 5. Shadow run — NOT STARTED
+
+Match against each batch, log what it *would* have cut, publish nothing. A week or two answers
+whether it finds advertising or accumulates junk, before one extra second is skipped.
+
+### 6. Publish matches, and unlock intro/outro — NOT STARTED
+
+Only once the shadow run has earned it.
+
+---
+
+## Measurements that must not be re-derived
+
+All in `docs/detection-findings.md` with full numbers. Re-running them costs hours.
+
+| finding | number |
+|---|---|
+| Ads found vs the declared length gap, current 12s floor | **~38%** median on shows with verified insertion |
+| Lowering the floor to 8s | LeVar 10:30 → 19:43, **Sleepy History 14:11 → 174:44** on a show with 0:48 of ads. **Do not lower it.** |
+| Filtering breaks by partner count or run length | Agreement drops 67% → 43%. **Disproven.** |
+| `MinRunSamples` and `MergeGapMs` | **Interact.** Low floor + 60s bridging welds noise into enormous spans. Never tune one alone. |
+| Ad repeats within a show | r = **0.877** |
+| Ad repeats across unrelated shows | r = 0.06–0.13 (two probes, both negative; one was a host-read spot that cannot repeat) |
+| Host-read ads (Darknet Diaries) | Never appear elsewhere. The library will not help such shows across shows, only across their own episodes over time. |
+| Episodes with no inserted ads at all | 138 of 322, and the app used to report "Ads inserted" on 111 of them. **Now 0.** |
+
+Peter's own domain knowledge, which has twice beaten my analysis: **ads come in ~15-second units,
+clustered in a few pods, with long stretches of content between.** That is what produced the 12s
+floor. And he reports ads repeat across podcasts within a network (NPR, Pushkin) — consistent with
+everything measured, though not directly caught in a probe.
+
+---
+
+## Traps that have already cost time
+
+- **Never run anything as root against `/site/listen.terst.org/data`.** An experiment did, left
+  `state.json` root-owned, and the service crash-looped **4,645 times over 13 hours** while
+  `systemctl is-active` reported "active" (it was the restart loop). Probe against a **copy** with
+  the audio dir symlinked, and check `/api/health`, not `is-active`.
+- **`gradle-test` reports a false FAIL for `connectedDebugAndroidTest`.** Line 70 scans
+  `*/test-results/*`; connected tests write to `app/build/outputs/androidTest-results/connected/`.
+  Parse that XML directly and check its mtime. **Unfixed in `/workspace/devtools`.**
+- **The emulator AVD is `doot_test_api36`.** There is no `feedreader_test` AVD. `adb` is at
+  `/opt/android-sdk/platform-tools/adb`, not on PATH.
+- **The pre-commit hook takes several minutes** (full build, tests, lint, detekt, coverage). Two
+  subagents have died mid-commit; one finished its work and never committed it. Always check
+  `git status` for orphaned work before assuming a failed agent did nothing.
+- **Post-hoc filtering cannot model a detector change.** Predicting the floor raise by filtering
+  already-detected breaks gave "31% → 34%"; the real answer was 30% → 23%. Measure by running the
+  detector, not by filtering its old output.
+
+---
+
+## Open, and genuinely Peter's
+
+- **Chapters** — settled by evidence, not asked: exactly one of his 61 shows publishes any, so
+  they live in the overflow.
+- **The queue screen** — asked, answered: *"Leave it."* It keeps its own screen.
+- **Still open: nothing from the UI proposal.** Both remaining questions were resolved — queue
+  stays its own screen ("Leave it"), chapters went to the overflow on evidence.
+- **Answered 2026-10-20:** the press skips as well as records, and the gesture that follows it
+  carries the rest of the label. See 1b.
+
+## Known gaps, deliberate
+
+- **Compose has no automated coverage** — no Robolectric. Everything visual is verified by eye on
+  the emulator only.
+- **The rejected-token notice is in-memory**, so after a process restart it reappears on the next
+  refresh rather than at launch.
+- **Library and Settings redesign (step 4 of the UI plan) not started.** Smallest wins; nothing
+  depends on it. Mockups in https://claude.ai/artifact/AnywWE7j7e96QUpXvF9yaW
+- **Intro/outro offsets still withheld.** They need sightings spanning more than one fetch
+  occasion — which step 4 of the library delivers.