listen.git/CLAUDE.md, branch main

Stop Apache writing the server's token into its access log

2026-11-05T22:48:23+00:00

/audio/ URLs carry the token as a query parameter, because a URL handed to a downloader has to be usable as it stands, and the "combined" log format logs %r -- the whole request line. So every audio fetch wrote the token in plaintext into a file logrotate keeps fourteen copies of: 335 lines across fifteen files, going back months. Both vhosts now use a listen_noquery format, %m %U %H instead of %r, which keeps the method, path and protocol and drops the query. Verified with a canary token against the live server rather than by reading the config: the request logged as "GET /api/health HTTP/1.1" with nothing after it, and a real request from the phone logged cleanly alongside it. The existing logs are redacted in place, gzipped ones included, with their lines and timestamps intact -- zero occurrences left, and none anywhere else under /var/log. The token itself is worth rotating, since it sat in those files for months. That is Peter's call. The real fix is for the audio URL to carry no token, which needs the downloader sending a bearer header for skip-server hosts -- and only those, or the token would go to every podcast CDN it fetches from. That app change has to be installed before the server stops emitting the query token, so it is not in this one. Config only; no code changed. Written up in CLAUDE.md so nobody restores "combined". Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Say what happened in words, and stop streaming from publishers

2026-11-05T08:42:28+00:00

Three things, from one red banner on Peter's phone reading "Playback failed: ERROR_CODE_IO_BAD_HTTP_STATUS". His words were "arcane and sketchy". Both halves were fair, and the second was pointing at a real bug rather than a wording problem. **The app was streaming episodes from their publishers.** `toMediaItem` falls back to the enclosure URL when there is no local file, and nothing stopped a play reaching it: press play on an undownloaded episode and it opened the publisher's CDN directly. Three things wrong with that. It is the automatic publisher fallback the download path had removed on purpose in October ("Do not reintroduce an automatic fallback for a transient failure"), surviving where nobody looked. It is an outbound request to a third party that nothing asked for -- the sketchy part, and Peter was right to read it that way. And worst, nothing in playback checks `audioFromServer`, so the breaks the server measured on *its* copy were being applied to a copy with different advertising in different places: skips landing in the wrong spot with no way to tell why. `PlaybackAdvancer.start` now returns NeedsFetch for any episode with no file, which routes through the download path that already honours the server rule. One caller still reaches the fallback -- Android Auto resolves a browse id straight to a media item without the advancer, and a car that silently plays nothing is worse than a car that streams. That needs fetching inside the session callback; until then OutboundRequestsTest pins the fallback to exactly one line. Ruled out first, with evidence rather than theory: the skip server (every request in its access log is a 200), the user agent (both the app's and Media3's get 200 from the enclosure), and my own deploy the day before. **The messages.** Media3's constant is now a sentence (`PlaybackFailureText`), and it was not the only one. The same pass found `Download failed: ${it.message}`, `Could not read that file: permission denied`, and `FeedFetcher` putting the literal word `UnknownHostException` under a podcast's name in the library -- that one reaches two surfaces, because the text is also stored as the subscription's `lastError`. **Two existing tests asserted the machine text came through**, which is how it survived review; they assert the opposite now. Accuracy turned out to be the harder half. "It will be retried" reads better than "Try the download button again" and is a lie: a failed download is deliberately left FAILED. A 404 and a 503 both mean "it didn't work" to a library and are different decisions for the person choosing whether to unsubscribe, so they say different things. Where the cause cannot be narrowed the sentence stays vague rather than guessing. Every cause goes to logcat -- which detekt caught me failing to do in FeedFetcher, three swallowed exceptions after I removed their messages. **The guardrails Peter asked for**, both watched failing before being trusted: - `PlainLanguageTest` -- no listener-facing message may be built out of an exception or spell out an error constant. Deliberately scoped to where errors surface rather than judging all prose. - `OutboundRequestsTest` -- one OkHttpClient, built in AppGraph, carrying SecureSchemeInterceptor; nothing may open a connection another way; and the publisher-stream fallback must stay at one site. The same arrangement the server has in safefetch/audit_test.go and for the same reason: this app fetches URLs chosen by podcast publishers. **And the readout, which Peter asked for directly.** The per-show "Detected skips" lines read the confirmation rows the on-device detector used to write, so since detection moved they could only ever say "Nothing found yet" -- on every show, including ones the server was publishing breaks for. Counted off the episodes now. Verified on a device against a staged state rather than argued: three of five episodes carrying a detected intro, one carrying an ad break, reading "Found in 3 of 5 episodes" / "None found in 5 episodes" / "Found in 1 of 5 episodes". *Not checked* and *checked, found nothing* are different answers again, which was the whole point of the readout. 1016 tests, detekt 0, lint clean. Coverage 94.58% -> 94.12%, COVERAGE_EXCEPTION for the same reason as yesterday: the deleted confirmation-row code was covered. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Delete on-device detection. Detection is the server's

2026-11-04T10:14:52+00:00

Peter's call: server first. The audio-fingerprint subsystem ran on the phone until detection moved to the server, and then kept being shipped: FingerprintWorker and MidrollWorker went at the time and the twelve classes they drove stayed, unreachable, in every APK since. ReachabilityTest found it on 2026-11-03 once it was widened to classes, and it has been sitting in that test's allowlist with its reasoning since. Gone: IntroOutroDetector, AudioCorrelation, MidrollScanner, MidrollDetector, MidrollUpdater, FingerprintUpdater, MediaCodecClipSource, Decimate, Fft, MonoResampler, PcmCodec, StoredClip, and the 67 unit tests over them. 3,092 lines. **Recoverable by name, not by archaeology.** The tag `on-device-detection` marks the last commit that has it, pushed to the local remote along with everything else from this session, and the tag message says how to bring it back. The work cost weeks and the approach may be wanted again. What stayed, because none of it is detection: SkipSummary words the readout, BreakTrack and BreakPopup draw and explain the timeline's marks, and MidrollClipStore shrank to deleteAllFor alone -- its write and read halves had one caller each and both were the deleted updater, but a phone that ran the old version still has clip files on disk and unsubscribing is the only thing that clears them. Its test now writes the old layout by hand, because that layout is a compatibility concern now rather than this class's own choice. IntroOutroDetector.CONFIRM_THRESHOLD and MidrollDetector.CONFIRM_THRESHOLD moved to SkipSummary, the only thing left that reads them. Both were three and the server's DefaultConfirmThreshold is also three, checked against each other before the move. **The database was deliberately not touched.** show_fingerprints, show_midrolls and episode_midrolls are the server's columns now -- ServerSkips writes its answers into exactly the columns the on-device detector used to, which is why playback needed no changes when detection moved. Four MidrollDao methods went because they were the discovery half (upsertCandidate, deleteCandidate, getUncheckedDownloaded, setMidrollCheckedAtMs) and the reachability check named them; a DAO method is not schema, so there is no migration here and nothing for a real library to go wrong over. Two things found while doing it, neither caused by it, both written up in CLAUDE.md: - **The per-show "Detected skips" readout can never say anything.** It reads candidate and confirmation rows, and nothing has written one since detection moved -- the server writes per-episode ranges and offsets instead. Verified on the emulator, not inferred: a fresh install with a live subscription shows "Nothing found yet / Nothing found yet / None found yet" while the server is finding and publishing breaks for that very show. It has been lying since the migration; deleting the detector only makes it undeniable. - **RetentionSweeper's pending-episode guard protects nothing**, for the same reason: nothing sets a pending basis any more, so both id sets are always empty. The APK barely moves -- 110KB of dex, 72.61MB to 72.50MB -- because it is dominated by the embedded Python. This was never going to be a size win and is not being sold as one; it is 3,092 fewer lines to read, search and mislead. 1004 tests pass, detekt 0, lint clean. Coverage 95.35% -> 94.58%, which is the documented COVERAGE_EXCEPTION case: deleting dead code that happened to be well covered. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Write down what the corpus actually says, and correct three stale claims

2026-11-02T07:51:10+00:00

The findings doc's last entry was 2026-10-19 and ended by naming what was missing: "a mechanism that can tell a real short ad from a short coincidence". Words are that mechanism, it was built, and thirty-two server commits later the doc never recorded what it measured. A future session would have re-attempted the ad/furniture classifier that was already measured and deleted. The new entry is derived from the live state file rather than quoted, which is how it caught that publish.go's own figure describes a subset. "92% to 104% across eighteen episodes" is the publisher-transcript number. Across the 24 episodes now publishing an ad span, recovery runs 20% to 146%. Split by where the master came from, the spread stops being a spread: a publisher's transcript gives 94-104%, a refetch gives a median of 33%. Nothing in Publishable looks at which it has, and refetch masters are now fifteen of those twenty-four -- the majority of a tier the design had decided not to act on. It also caught me getting a conclusion wrong. Two Reply All episodes publish ad spans at 146% and 138% of their length gap, which reads as the detector over-claiming by half a minute of programme. Looking at the show instead of the episode: Reply All's gaps run from -0.48 to +1.02 minutes across eight episodes. They straddle zero, so the feed's declared durations are not a baseline for that show and the thing being exceeded is noise. The structural finding is the useful one -- HasInsertedAdvertising asks only whether this episode's gap clears one ad unit, and has no notion of whether the show's gaps are consistent enough to mean anything. Both versions are in the entry, because the wrong one is the one a reader would reach for next time. Nothing here changes behaviour. Whether a refetch should publish an ad at all is Peter's call and is written up as such. CLAUDE.md, separately: - It still said the episode description shows about two lines. That was changed to ten yesterday, for a reason that is recorded: the limit existed to stop long notes pushing the transport off-screen, and the merge had already put the notes below the transport. Left as it was, the doc was an instruction to reintroduce the bug. - "207 unit tests" has been wrong for most of the project's life; the suite is past nine hundred. Replaced with a note to ask the wrappers, since any number written here goes stale the same way. - The TDD rule banned backfilling tests onto existing code without qualification, which also forbids the one case where it is right -- pinning behaviour before moving it, so a refactor is provably behaviour-preserving. Narrowed to say that, and to say the order matters. The sixteen formatter tests in 2cc252a were written that way. - Added the rule those formatter files were moved to satisfy: anything the app says out loud lives in a named file of its own and is tested, never inside the composable that renders it. It was already followed by eight files and stated only in WaitingForServerText's own comment. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Gate the server too, and stop reading go test with grep

2026-10-31T18:16:41+00:00

The commit hook had nine steps and not one of them touched the server. Every server change for a week -- twenty-five commits, including the ad detectors and the thing that publishes skips -- went in on somebody remembering to type "go test" and reading the output. That is exactly the habit the Gradle wrappers exist to replace, in the half of the repo nobody had wrapped. go-verify reports from exit codes and parsed `go test -json`: gofmt, build, vet and a real test count. It was watched catching both failures that look like success in a terminal -- a failing test, named; and a package that fails to compile, where go test exits non-zero with no failing test and the count silently drops from 366 to 356. 366 is a number I did not have until now, because "go test ./... | grep -v ok" cannot produce one. It runs first in the hook, because ten seconds before the Android build beats several minutes after it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Write down the transcript pipeline, and that it is verified on a device

2026-10-31T01:00:19+00:00

CLAUDE.md still described the server as doing audio detection and nothing else. Six days of work later it recognises speech, subtracts transcripts, and publishes the result as skips, and none of that was written down anywhere a future session would look. The device verification is the part worth recording most. Heavyweight's "Jonathan + Death of an Artist", breaks at 0-1:45 and 13:32-15:16: downloaded from the server, both rows landed in episode_midrolls, twenty seconds of playing put the position at 2:03 -- which is only reachable by jumping the pre-roll -- and the post-roll skipped to the end, marked the episode played and reset the position. Server to phone to playback, on hardware, not inferred from the fact that each link has a passing test. Also written down: the measured 10.3x realtime, the four alignment mistakes that only measurement caught, why a declined transcript is kept rather than discarded, why refetching belongs to slow-drip shows, and the shared-text signal that is prototyped but not built. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Make the episode screen the player and delete the separate player screen

2026-10-19T09:31:21+00:00

Opening an episode now puts you in its transport at the stored position; pressing play changes nothing about the layout. One primary control (Play/Resume/Pause) with back/forward, a speed chip that opens the four offered speeds (stored 1.5/1.75 still play and are named on the chip), and everything secondary behind a top-bar overflow: queue, download, played, watch, mark-older, chapters, and the publisher-copy action (behind a dialog carrying its warning). Description shows about two lines, the rest on tap. The now-playing bar, library bar, queue, share receiver and nav graph all open the playing episode's screen. PlayerViewModel stays the single owner of playback state; no playback decision moved into the UI. PlayerScreen.kt and WatchButton.kt are deleted. Coverage drop, if any, is from removing covered dead code. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Record that lowering the noise floor to 8s is wrong

2026-10-19T09:15:03+00:00

The 12s floor cut real detection on shows with verified ads -- LeVar Burton Reads fell from 19.0 to 10.5 minutes found against a 26.6 minute gap -- so 8s looked like the fix. Measured on an isolated copy of the live data it is not: Sleepy History at 8s finds 174 minutes of "advertising" in a show that has 48 seconds of it, worse than the 4s floor that produced Peter's original complaint. MinRunSamples and MergeGapMs interact and cannot be tuned separately. A low floor lets noise fragments survive and the 60s bridging then welds them into enormous spans. Had only LeVar been measured the floor would have been lowered. Also re-pins .coverage-baseline, which the hook left modified after d96c876 deleted covered readout code. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Record that the silent publisher fallback is gone

2026-10-17T07:55:17+00:00

CLAUDE.md still described it as deliberate design with its original rationale, which would have had a future session reintroduce the exact behaviour Peter asked to remove. Also records the three cases that must keep downloading directly, why a rejected token is permanent rather than transient, and what audioFromServer is for. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Examine each batch once instead of every poll

2026-10-07T09:14:11+00:00

The first full pass over all 42 feeds finished, then the server kept re-deriving the same answers: Radiolab reported "53:29 of 59:39" four times in one day. pollOne called detect unconditionally and the mirror skips files it already has, so nothing had changed between passes. At ~6.5 minutes a show that is 4.5 hours a cycle against a two-hour poll interval -- the machine never finished a cycle and never idled, burning 13h42m of CPU in 11h of wall clock, roughly 60% of it on the intro search whose results are deliberately not published. Show.DetectedBatch records the episode ids a completed pass examined. The batch is the unit rather than the episode because every offset and every break comes from comparing an episode against the others fetched alongside it, so one arrival or one pruning legitimately changes all of them. Order-independent, since feeds are not reliably sorted. Recorded only on completion, so a cancelled or failed pass is retried. Episode.DetectedAtUnix could not serve as the gate: it is only set for episodes appearing in result.Offsets, so an episode nothing was found in never got a timestamp and the gate would never close. Also records what the first full corpus showed, including a disproven hypothesis worth not retrying -- filtering breaks by partner count or run length looks obviously right and makes agreement with the declared length gap substantially worse -- and a correction: Sleepy History's 117 minutes of "advertising" was a show-level sum sitting almost entirely inside the two episodes its own guard already refused. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B