listen.git/server, branch main

The audio URL stops carrying the token

2026-11-11T16:30:54+00:00

Apache logs the request line, so a token in the query string was a copy of the secret in every line of a file logrotate keeps fourteen of. That is why the vhosts log /audio/ without query strings at all; this removes the reason they had to. Safe now because both halves of the phone send the header: the downloader always did, and the player does since it got a ResolvingDataSource. Verified against this server before changing it -- the player reaches READY on listen.terst.org audio with a header and no token in the URL. Query tokens are still *accepted*. Withdrawing that as well would lock out an older build with no way back in, and accepting one costs nothing now that nothing emits it. The transcript URL made this move already, for the same reason. This was the last one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Shows have a category, not a tier

2026-11-11T08:54:52+00:00

Tier said "how important" and the shows needed "what kind". Seven ordered categories replace it: news, analysis, curiosity, sailing, true stories, fiction, deep dives. The order is still the server's processing priority, so one field carries both axes -- which is the idiosyncrasy docs/show-kinds.md is written to watch, with four signs that a better abstraction has arrived. All 42 known feeds are refiled by migration 20 -> 21, which rebuilds the subscriptions table rather than renaming the column: RENAME COLUMN keeps the old 'ordinary' DEFAULT and fails schema validation, and DROP COLUMN needs SQLite 3.35 against minSdk 29's 3.28. The wire value for true stories is "truestory", not "true", because SQLite reads an unquoted true in a DEFAULT as the boolean and stores 1. Schema 21 is a new exported JSON; migrate20To21RenamesTierAndRefilesTheKnownShows covers it, and all 22 device migration tests pass. The coverage exclusion for the settings composable follows the file's rename; left pointing at TierSettingKt it read as a 0.6% drop. The remaining 0.08% is the deletion of the covered tier-seeding code, taken as an exception. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

The car plays our copy too, and the transcript URL stops leaking the token

2026-11-05T09:09:50+00:00

**Android Auto was the one caller still streaming from publishers.** `onAddMediaItems` resolved a browse id straight to a media item without asking whether the audio existed, so an undownloaded episode was handed to the player as its publisher's URL. That is the worst place for it: the car is where most of this library is actually heard, and misplaced skips are least obvious there. It goes through `PlaybackAdvancer.startFetchingIfNeeded` now -- start, and fetch first if there is no file -- the same decision the in-app paths make, in the one class tested for it. A fetch that fails stays failed rather than falling back: the publisher's copy carries different advertising, so the server's measured breaks land in the wrong place, and silence that can be retried beats skipping into the programme. With that, nothing reached the fallback, so it is gone. `setUri(localPath ?: mediaUrl)` is now `setUri(localPath)`, `MediaUrl` is deleted, and OutboundRequestsTest asserts zero remote media items rather than pinning one. `start` also reports Gone for an episode whose show *and* file have both gone, which is what made the fallback removable -- an orphan that still has its file plays. **Two findings from the security review of the last commit, both mine, both real:** - `Request.Builder().url()` throws on a URL it cannot parse and only IOException was caught, so a feed with a junk enclosure threw straight past the Result its caller folds. Survivable when this ran from a worker; not now that every play goes through a fetch, where one bad enclosure in one feed would stop any episode starting. Caught in the downloader, and `fetchAndStart` turns a thrown fetch into FetchFailed -- rethrowing CancellationException, which a test now pins, because swallowing it would report a cancelled start as a failed one and leave a false error on screen. - `FeedFetcher` logged whole feed URLs. Private feeds carry a token in the query and some carry credentials in the userinfo. `FeedFailureText.redact` cuts them to scheme and host, and is tested as the security control it is rather than as tidiness. The unparsable branch logs neither the address nor the exception, and says why: that exception's message *is* the address, so logging the cause would put back exactly what redacting removes. **The transcript URL no longer carries the token.** Apache logs the request line, so every /api/transcript fetch wrote the server's token into its access log in plaintext, across fourteen rotations. It was redundant there: the phone sends Authorization: Bearer anyway and tokenOK takes either. The audio URL still carries one -- not redundant, the downloader has no credentials of its own -- so removing that needs an app change installed before the server stops emitting it. The gate refused this twice and was right both times. The coverage drop was not all deletion: the redaction and the cancellation contract were genuinely untested, and have tests now. Then detekt caught an import left behind by moving the redaction into FeedFailureText -- a move I made after the last time I ran it. 1031 app tests, 376 server tests, detekt 0, lint clean. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Keep what the listener asked for, and publish ads only from a publisher's master

2026-11-03T08:22:11+00:00

Two changes to the server, both measured against the live data rather than reasoned about. **Subtraction publishes only from a publisher's transcript.** Subtracting a publisher's transcript recovers 94-104% of the advertising the file lengths imply; subtracting a refetch -- a second download of the same assembled file, carrying its own ads in its own places -- recovers a median of 33%. That tier had already been tried and rejected at 21% and 0%, but nothing in the publish path was told which master it had, so it reached the phone anyway on seventeen ad spans. Written as a denylist first, and measuring it against a copy of the live data showed why that was the wrong shape: sixteen episodes carry a master whose label is blank -- the field postdates the first masters written, and Reclaim restored those files after the library was destroyed with the blank intact -- and they published on the strength of not being the one named bad kind. They happened to be publisher transcripts, so the answer was right by luck. It is an allowlist now, and the blank labels are repaired from the master files, which have said "published" all along. Measured on a copy, not predicted: before 49 ad spans (17 refetch, 24 unlabelled, 8 published) + 604 cue, 245 episodes after 62 ad spans (all publisher-master) + 611 cue, 245 episodes No episode loses its skips; the refetch ones trade advertising nobody could measure for the show's own recurring audio, which is a preference rather than an unconditional skip. Measuring also caught the recurring pass blanking the refusal subtraction had just written -- both passes answer "why is nothing published" through one field and recurring runs second, so all thirty-two episodes that now carry a reason were losing it a moment later. **Being asked for keeps an episode mirrored.** Retention was publication date alone, and pruning calls words.Forget, which deletes the transcript *file*. So an episode outside the newest eight was fetched on demand, recognised, served, and deleted on the next poll -- and asking again paid the whole cost over. From the server's own journal over twenty-one days: 397 transcription runs covering 309 distinct episodes, with 7.7 of the 33.4 hours of whisper time spent on episodes it had already recognised and thrown away. 23% of the budget, re-doing work it had discarded. Working through a back catalogue -- the archive tier, the slow-drip shows, the listening this was built for -- therefore paid full recognition for every episode and again to resume one. The window still decides what is worth fetching; RequestedAtUnix now decides what is worth keeping, stamped on every /api/episode call including for episodes already mirrored, because resuming something played last month must not let it fall out mid-listen. Kept for 28 days after the last ask, the same span refetchAfterSeconds uses, being the same kind of judgement about how long something stays interesting. Disk points the same way: transcripts are 36MB for 445 files, audio is 17GB for 336 episodes, and the volume has 365GB free. The expensive artefact is the small one. 374 server tests pass. Not yet deployed; the phone's half of this -- a pending reply is retried only on a feed refresh -- is still to come. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Write down what the corpus actually says, and correct three stale claims

2026-11-02T07:51:10+00:00

The findings doc's last entry was 2026-10-19 and ended by naming what was missing: "a mechanism that can tell a real short ad from a short coincidence". Words are that mechanism, it was built, and thirty-two server commits later the doc never recorded what it measured. A future session would have re-attempted the ad/furniture classifier that was already measured and deleted. The new entry is derived from the live state file rather than quoted, which is how it caught that publish.go's own figure describes a subset. "92% to 104% across eighteen episodes" is the publisher-transcript number. Across the 24 episodes now publishing an ad span, recovery runs 20% to 146%. Split by where the master came from, the spread stops being a spread: a publisher's transcript gives 94-104%, a refetch gives a median of 33%. Nothing in Publishable looks at which it has, and refetch masters are now fifteen of those twenty-four -- the majority of a tier the design had decided not to act on. It also caught me getting a conclusion wrong. Two Reply All episodes publish ad spans at 146% and 138% of their length gap, which reads as the detector over-claiming by half a minute of programme. Looking at the show instead of the episode: Reply All's gaps run from -0.48 to +1.02 minutes across eight episodes. They straddle zero, so the feed's declared durations are not a baseline for that show and the thing being exceeded is noise. The structural finding is the useful one -- HasInsertedAdvertising asks only whether this episode's gap clears one ad unit, and has no notion of whether the show's gaps are consistent enough to mean anything. Both versions are in the entry, because the wrong one is the one a reader would reach for next time. Nothing here changes behaviour. Whether a refetch should publish an ad at all is Peter's call and is written up as such. CLAUDE.md, separately: - It still said the episode description shows about two lines. That was changed to ten yesterday, for a reason that is recorded: the limit existed to stop long notes pushing the transport off-screen, and the merge had already put the notes below the transport. Left as it was, the doc was an instruction to reintroduce the bug. - "207 unit tests" has been wrong for most of the project's life; the suite is past nine hundred. Replaced with a note to ask the wrappers, since any number written here goes stale the same way. - The TDD rule banned backfilling tests onto existing code without qualification, which also forbids the one case where it is right -- pinning behaviour before moving it, so a refactor is provably behaviour-preserving. Narrowed to say that, and to say the order matters. The sixteen formatter tests in 2cc252a were written that way. - Added the rule those formatter files were moved to satisfy: anything the app says out loud lives in a named file of its own and is tested, never inside the composable that renders it. It was already followed by eight files and stated only in WaitingForServerText's own comment. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Give the ad slot's length one home

2026-11-02T07:42:12+00:00

Four packages each wrote 15 seconds for the length of an advertising slot -- skips, transcript twice, and signature -- and every copy carried a comment saying it agreed with the others: "the same unit skips.Classify uses", "the same figure the length-gap rule uses", "one ad unit from the start". The agreement was real and deliberate, and recorded only in prose, so a corpus that revised the figure would have moved one copy and left three disagreeing silently. internal/advertising holds it once. Each site keeps its own name and its own explanation of what the unit means there, because those genuinely differ; only the number is shared. The consolidation on its own fixes the four copies that existed. The audit test refuses the fifth: it walks the server's production source and fails on the number written as a literal, in either of Go's spellings. Watched failing on both before being trusted. Same arrangement as safefetch/audit_test.go, and for the same reason -- the instance that was noticed was never the whole of it. No behaviour change: 366 server tests pass, same as before. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Stop a second device from emptying the library, and recover what one did

2026-10-31T09:32:15+00:00

I pointed the emulator at the live server to verify that published breaks actually skip. It worked. It also pushed the emulator's own four-show subscription list, which replaced the real one, and at 07:07:13 the server reclaimed every show that device had never heard of: 26 shows and 290 mirrored episodes, deleted in a second. Three fixes, in the order they matter. **The list can no longer be emptied by one push.** It is still replaced wholesale -- the phone owns it, and reconciling two editable copies of something changed this rarely costs more in sync bugs than it saves -- but a push that keeps fewer than half of ten or more feeds is refused with a 409 and logged. Losing most of a library in one push is a second device or a bug; it is never somebody unsubscribing. Below ten feeds there is nothing to protect: a new phone's first push legitimately replaces nothing with everything. **Recognition is recovered rather than redone.** A transcript is keyed by the episode id, which comes from the feed and the guid, so it outlives the record and matches again when the episode is mirrored afresh. 332 transcript files survived on disk. Reclaim restores them -- 18 immediately, the rest as re-mirroring reaches them -- instead of spending twenty hours of CPU recognising text the server is already holding. It refuses a transcript whose last cue runs past the end of the audio, which is the same check made of a publisher's. **The pipeline's Save was never wired**, which the security review caught before it could panic in anger: the extraction moved the call and not the field. It is now optional, because every test wants it nil, and assumed-non-nil is what shipped a server that would have died on its first recognised episode. Reclaim runs before each show is polled as well as at startup, or a re-mirrored episode would wait for a restart to find the words already sitting next to it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Deep clean, part one: get the pipeline out of the command

2026-10-31T09:16:45+00:00

cmd/listend/main.go had reached 1,249 lines carrying nine passes, and cmd has no test file -- so every judgement in the transcript pipeline lived where nothing could reach it. That is not hypothetical: a refetch master blocked the recurring detector for a fortnight because the condition was wrong in a place no test could see. internal/words now holds the five passes with judgement in them -- Harvest, Recognise, Rederive, Recurring, SecondCopy -- behind a Pipeline that names its dependencies. main.go is 631 lines and does what a command should: read flags, wire things together, serve. **Moving it found a latent bug.** Pruning an episode's audio cleared its transcript fields but not its master ones, so a re-mirrored episode would claim a master whose file had been deleted -- a second description it could not produce, and subtraction finding nothing for it for ever. Exactly the shape of the bug that had just cost a fortnight. Pipeline.Forget now clears all of it, including the attempt timestamps, so a re-mirrored episode is done again rather than skipped for ever. Pinned by a test that asserts it needs recognising afterwards. Also in this part: Gradle's file-system watching is off. Chaquopy extracts a 3,200-file Python environment into app/build mid-build and the watcher tripped over re-registering it, printing "Caught exception: Already watching path" in front of every build error this project produced all week. It costs nothing to turn off -- measured twice each on a no-op compileDebugKotlin, 1.50s and 1.59s with, 1.46s and 1.46s without -- because that extraction invalidates the VFS anyway. A watcher holding stale state about a tree another tool rewrites causes the "clean and try again" class of problem rather than curing it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Shows get a priority, so the news is not stuck behind a back catalogue

2026-10-31T08:39:44+00:00

Peter's framing: a few shows are news and stop being worth hearing within a day, a tier he keeps up with, and a long tail that is not topical at all. It matters because recognising one episode costs the server about five minutes, so a five-hour back-catalogue episode arriving the same morning as the news takes most of a cycle before the news is looked at. Three tiers and no more. A number would invite fiddling and would not mean anything; these are the three distinctions that were described. The tier rides beside the subscription list rather than inside it, so a server that knows nothing about tiers reads the list exactly as before and a phone that knows nothing keeps pushing what it always did. Only the shows placed somewhere other than the default are sent -- "ordinary" for sixty shows is the same information as nothing, in sixty times the space. It orders both the poll and the recognition queue. Recognition now sorts on three things in the order they matter: a topical show first, then an episode whose answer checks itself against the length gap, then newest. **A tier only exists for a subscribed feed**, which a test caught: the list and the tiers arrive in one push, and setting the tiers after the list put back a placement for a feed that same push had just removed. A resubscription months later would have inherited it. Unrecognised tiers are ignored at both ends and read as the default, which is the one that starves nothing -- the same rule the label store follows for a verdict it does not know. Schema 20, migrated and checked on a device. The control says what it does not do as well as what it does: nothing about playback changes, and a show placed anywhere still plays, downloads and skips identically. A setting that looks like it changes playback and does not is worse than no setting. The coverage gate caught the push body having no test, which was fair -- it is hand-built JSON going over the wire, where a missing brace is a library that silently stops being mirrored. Its exact output is now pinned, including a URL with a quote in it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

A break says which detector found it, and the episode screen says so

2026-10-31T07:03:03+00:00

Three detectors now cut audio out of episodes and the app said nothing about which. The listener judging a skip wants to know what judged it, and they are not equally sure of themselves: one compares waveforms across a batch of episodes, the other reads what was said and checks itself against the declared length. A break both found is marked as both, because agreement by entirely different means is the strongest thing either can offer. "Found by the audio and the words", under the numbers and quieter than them -- it explains them rather than competing with them. Schema 19. Existing rows default to the audio scan because that is where every break in the database came from; the words could not produce one until this week. "Unknown" would have been honest about the column and useless about the rows. A server that says nothing gets the same default, for the same reason. The coverage gate caught the new field's default never being exercised, which was fair: every call site passes it, and the case it exists for is an episode with nothing cut and nothing to account for. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B