listen.git/server/cmd, branch main

Shows have a category, not a tier

2026-11-11T08:54:52+00:00

Tier said "how important" and the shows needed "what kind". Seven ordered categories replace it: news, analysis, curiosity, sailing, true stories, fiction, deep dives. The order is still the server's processing priority, so one field carries both axes -- which is the idiosyncrasy docs/show-kinds.md is written to watch, with four signs that a better abstraction has arrived. All 42 known feeds are refiled by migration 20 -> 21, which rebuilds the subscriptions table rather than renaming the column: RENAME COLUMN keeps the old 'ordinary' DEFAULT and fails schema validation, and DROP COLUMN needs SQLite 3.35 against minSdk 29's 3.28. The wire value for true stories is "truestory", not "true", because SQLite reads an unquoted true in a DEFAULT as the boolean and stores 1. Schema 21 is a new exported JSON; migrate20To21RenamesTierAndRefilesTheKnownShows covers it, and all 22 device migration tests pass. The coverage exclusion for the settings composable follows the file's rename; left pointing at TierSettingKt it read as a 0.6% drop. The remaining 0.08% is the deletion of the covered tier-seeding code, taken as an exception. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Keep what the listener asked for, and publish ads only from a publisher's master

2026-11-03T08:22:11+00:00

Two changes to the server, both measured against the live data rather than reasoned about. **Subtraction publishes only from a publisher's transcript.** Subtracting a publisher's transcript recovers 94-104% of the advertising the file lengths imply; subtracting a refetch -- a second download of the same assembled file, carrying its own ads in its own places -- recovers a median of 33%. That tier had already been tried and rejected at 21% and 0%, but nothing in the publish path was told which master it had, so it reached the phone anyway on seventeen ad spans. Written as a denylist first, and measuring it against a copy of the live data showed why that was the wrong shape: sixteen episodes carry a master whose label is blank -- the field postdates the first masters written, and Reclaim restored those files after the library was destroyed with the blank intact -- and they published on the strength of not being the one named bad kind. They happened to be publisher transcripts, so the answer was right by luck. It is an allowlist now, and the blank labels are repaired from the master files, which have said "published" all along. Measured on a copy, not predicted: before 49 ad spans (17 refetch, 24 unlabelled, 8 published) + 604 cue, 245 episodes after 62 ad spans (all publisher-master) + 611 cue, 245 episodes No episode loses its skips; the refetch ones trade advertising nobody could measure for the show's own recurring audio, which is a preference rather than an unconditional skip. Measuring also caught the recurring pass blanking the refusal subtraction had just written -- both passes answer "why is nothing published" through one field and recurring runs second, so all thirty-two episodes that now carry a reason were losing it a moment later. **Being asked for keeps an episode mirrored.** Retention was publication date alone, and pruning calls words.Forget, which deletes the transcript *file*. So an episode outside the newest eight was fetched on demand, recognised, served, and deleted on the next poll -- and asking again paid the whole cost over. From the server's own journal over twenty-one days: 397 transcription runs covering 309 distinct episodes, with 7.7 of the 33.4 hours of whisper time spent on episodes it had already recognised and thrown away. 23% of the budget, re-doing work it had discarded. Working through a back catalogue -- the archive tier, the slow-drip shows, the listening this was built for -- therefore paid full recognition for every episode and again to resume one. The window still decides what is worth fetching; RequestedAtUnix now decides what is worth keeping, stamped on every /api/episode call including for episodes already mirrored, because resuming something played last month must not let it fall out mid-listen. Kept for 28 days after the last ask, the same span refetchAfterSeconds uses, being the same kind of judgement about how long something stays interesting. Disk points the same way: transcripts are 36MB for 445 files, audio is 17GB for 336 episodes, and the volume has 365GB free. The expensive artefact is the small one. 374 server tests pass. Not yet deployed; the phone's half of this -- a pending reply is retried only on a feed refresh -- is still to come. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Stop a second device from emptying the library, and recover what one did

2026-10-31T09:32:15+00:00

I pointed the emulator at the live server to verify that published breaks actually skip. It worked. It also pushed the emulator's own four-show subscription list, which replaced the real one, and at 07:07:13 the server reclaimed every show that device had never heard of: 26 shows and 290 mirrored episodes, deleted in a second. Three fixes, in the order they matter. **The list can no longer be emptied by one push.** It is still replaced wholesale -- the phone owns it, and reconciling two editable copies of something changed this rarely costs more in sync bugs than it saves -- but a push that keeps fewer than half of ten or more feeds is refused with a 409 and logged. Losing most of a library in one push is a second device or a bug; it is never somebody unsubscribing. Below ten feeds there is nothing to protect: a new phone's first push legitimately replaces nothing with everything. **Recognition is recovered rather than redone.** A transcript is keyed by the episode id, which comes from the feed and the guid, so it outlives the record and matches again when the episode is mirrored afresh. 332 transcript files survived on disk. Reclaim restores them -- 18 immediately, the rest as re-mirroring reaches them -- instead of spending twenty hours of CPU recognising text the server is already holding. It refuses a transcript whose last cue runs past the end of the audio, which is the same check made of a publisher's. **The pipeline's Save was never wired**, which the security review caught before it could panic in anger: the extraction moved the call and not the field. It is now optional, because every test wants it nil, and assumed-non-nil is what shipped a server that would have died on its first recognised episode. Reclaim runs before each show is polled as well as at startup, or a re-mirrored episode would wait for a restart to find the words already sitting next to it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Deep clean, part one: get the pipeline out of the command

2026-10-31T09:16:45+00:00

cmd/listend/main.go had reached 1,249 lines carrying nine passes, and cmd has no test file -- so every judgement in the transcript pipeline lived where nothing could reach it. That is not hypothetical: a refetch master blocked the recurring detector for a fortnight because the condition was wrong in a place no test could see. internal/words now holds the five passes with judgement in them -- Harvest, Recognise, Rederive, Recurring, SecondCopy -- behind a Pipeline that names its dependencies. main.go is 631 lines and does what a command should: read flags, wire things together, serve. **Moving it found a latent bug.** Pruning an episode's audio cleared its transcript fields but not its master ones, so a re-mirrored episode would claim a master whose file had been deleted -- a second description it could not produce, and subtraction finding nothing for it for ever. Exactly the shape of the bug that had just cost a fortnight. Pipeline.Forget now clears all of it, including the attempt timestamps, so a re-mirrored episode is done again rather than skipped for ever. Pinned by a test that asserts it needs recognising afterwards. Also in this part: Gradle's file-system watching is off. Chaquopy extracts a 3,200-file Python environment into app/build mid-build and the watcher tripped over re-registering it, printing "Caught exception: Already watching path" in front of every build error this project produced all week. It costs nothing to turn off -- measured twice each on a no-op compileDebugKotlin, 1.50s and 1.59s with, 1.46s and 1.46s without -- because that extraction invalidates the VFS anyway. A watcher holding stale state about a tree another tool rewrites causes the "clean and try again" class of problem rather than curing it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Shows get a priority, so the news is not stuck behind a back catalogue

2026-10-31T08:39:44+00:00

Peter's framing: a few shows are news and stop being worth hearing within a day, a tier he keeps up with, and a long tail that is not topical at all. It matters because recognising one episode costs the server about five minutes, so a five-hour back-catalogue episode arriving the same morning as the news takes most of a cycle before the news is looked at. Three tiers and no more. A number would invite fiddling and would not mean anything; these are the three distinctions that were described. The tier rides beside the subscription list rather than inside it, so a server that knows nothing about tiers reads the list exactly as before and a phone that knows nothing keeps pushing what it always did. Only the shows placed somewhere other than the default are sent -- "ordinary" for sixty shows is the same information as nothing, in sixty times the space. It orders both the poll and the recognition queue. Recognition now sorts on three things in the order they matter: a topical show first, then an episode whose answer checks itself against the length gap, then newest. **A tier only exists for a subscribed feed**, which a test caught: the list and the tiers arrive in one push, and setting the tiers after the list put back a placement for a feed that same push had just removed. A resubscription months later would have inherited it. Unrecognised tiers are ignored at both ends and read as the default, which is the one that starves nothing -- the same rule the label store follows for a verdict it does not know. Schema 20, migrated and checked on a device. The control says what it does not do as well as what it does: nothing about playback changes, and a show placed anywhere still plays, downloads and skips identically. A setting that looks like it changes playback and does not is worse than no setting. The coverage gate caught the push body having no test, which was fair -- it is hand-built JSON going over the wire, where a missing brace is a library that silently stops being mirrored. Its exact output is now pinned, including a URL with a quote in it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Score the detectors against what the listener heard, and fix two things it found

2026-10-31T06:36:34+00:00

Every other number in this project comes from the declared-versus-served length gap, which says how much advertising a copy carries and nothing about where. A detector could find the right total in the wrong places and score 100%. The position check I wrote earlier used the same signal that builds the candidates, so it was partly circular and structurally blind to the failure it most needed to see. The 19 labels are the only statement about *where* that no detector produced: each is Peter pressing a button at a moment in an episode. listen-score is read-only, takes the state file, and prints how the published breaks did. **Its first answer was wrong, and checking rather than reporting it caught that.** It said 2 of 18 advertisements were covered, which would have meant the detectors were far worse at position than at totals. They are not: breaks at 829-999 and 1360-1519 against labels at 853-1000 and 1489-1540 -- correct detections, scored as complete misses because the break stopped a second short of the label's end. Twelve of the nineteen labels have an end the store itself documents as a guess, "where a jump happened to land". Scoring against a guessed end is scoring against noise. A label's start is solid; its end counts only when measured. 5 of 18 after that. **A refetch master was blocking the better detector.** Tangle's "The Israeli elections" has a 229-second gap, a master from a second fetch that carried the same campaign, and therefore no subtraction candidates at all -- and findRecurring skipped it because it had a master. Treating "has a master" as "has a better answer" is wrong when the master is useless. It now skips only episodes that actually published something. **The one programme cut is the audio scan's, not the words'.** Its break at 696-708 sits exactly on the stretch Peter said was programme. Worth knowing before anything else gets blamed for it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Find what a show says every week, and publish it as the show's own audio

2026-10-31T03:59:56+00:00

The text analogue of the audio scan, and it costs nothing: the server already holds transcripts for eight episodes of each show, mirrored within a minute of each other, carrying one campaign and eight different programmes. Text several of them share is not the programme. It reaches where subtraction cannot. Subtraction needs a second description of one episode -- a publisher's transcript or a second fetch -- and 18 of 337 mirrored episodes have one. This needs only the episodes already held. Measured against the length gap on the live library: Heavyweight 111%, Incels 105%, Ezra Klein 92%, Tangle 83%, against refetching's 21% on the same show. **It is published as the show's own audio, not as advertising, and that is a measured limit rather than caution.** What recurs is a show's sponsor reads and its furniture together -- its welcome, its break cue, its credits -- and I tried to separate them and could not. Scored against the 404 recurring stretches in episodes that also hold a publisher's master, which says which is which: ad copy markers 5% of the advertising, 4% of the furniture length mean 51s against 29s, overlapping throughout position ads 62 open / 176 mid / 41 close furniture 12 open / 61 mid / 52 close A first classifier built on ad copy agreed with the master 33% of the time and called 263 advertisements furniture. None of those signals separates the two. So the detector claims only what it measured: this is text the show says every week. That is what KindCue means, and it leaves the listener's setting in charge -- a show's own audio is skipped by preference, advertising unconditionally, and claiming the stronger of the two on a coin flip would be the worse mistake. Refused when more than 35% of an episode recurs, the same share the ad scan refuses at: a re-aired episode or a formulaic segment would otherwise be eaten whole. Guarded by share rather than by the length gap, because the episodes this is worth most on are the ones with no gap at all -- a show that bakes its reads into what it publishes declares the length it serves, so the gap is blind to every advertisement in it. Skipped entirely for an episode that already has measured advertising. The better answer must not be overwritten by the vaguer one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Decide publishing every time, not only when the candidates moved

2026-10-30T08:42:51+00:00

The re-derive pass skipped an episode whose advertising had not changed since last time, which is sensible for rewriting the file and wrong for everything else: what is *fit to act on* is a separate judgement from what was found, and it changes on its own -- it just did, when publishing was added. So the eighteen episodes that had been measured for days published nothing, because their candidates were already correct. Found by looking at the state file after deploying rather than by assuming the deploy had worked. The readout now says how many episodes publish how many breaks, every start, so "nothing changed" and "nothing is published" can no longer look alike. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Publish the advertising the words measured, and only that

2026-10-30T08:37:15+00:00

Subtraction has earned it. Eighteen episodes, 92% to 104% of what the declared-versus-served length gap implies, every pod found, edges pulled back off the programme, and the one class of mistake that mattered -- a break opening on the host's own sentence -- traced and fixed. **Only subtraction is published.** Reading the ad copy measured 0%, 0%, 15% and 29% on the same kind of episode, and on a show that bakes its reads in it found 2:13 where the lengths can see nothing at all. That stays a hint beside the words, dimmed on the transcript screen, and never becomes a seek. **The length gap is the guard as well as the measure.** It comes from the feed and the file rather than from any detector, so it is the one number able to contradict one. Nothing is published for an episode with no gap -- there would be nothing to check it against -- and a candidate set claiming half again as much as was inserted is refused with its reason, because it has found something other than advertising. They land in their own field rather than beside the audio scan's, which rewrites its list from scratch on every detection pass and would wipe them. The API serves the union, merging ranges that overlap or touch: playback seeks to a break's end, so two overlapping breaks would seek out of the first and straight into the second, and two touching ones would resume for no time at all before skipping again. Advertising wins where merged ranges disagree about what they are -- the phone skips advertising unconditionally and a show's own audio only by preference, and being wrong the other way means sitting through an advertisement the server had already found. Nothing about playback changes. They arrive in the field the phone has read since the on-device detector existed. **Refetching is now gated on age**, which is Peter's steer and what the numbers say. A second fetch made minutes after the first came back twenty seconds different and found 21% of the advertising; another came back the same length to the second and found none. Insertion varies the amount per request, not the campaign. So it is for shows whose episodes are still worth hearing four weeks later -- a slow-drip archive, a leisure show -- and not for daily news, which is pruned long before a campaign rotates. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Tell the phone what reading the words concluded

2026-10-30T08:25:35+00:00

Three numbers on the line that already says an episode has a transcript: how many stretches of advertising were found, how long they come to, and how. The method travels with them because subtraction and ad-copy reading are not close in quality -- 92-104% of the length gap against 0-29% -- and a figure on its own would invite reading the two alike. An episode that turned up nothing says nothing rather than zero, which is the same three-state care the rest of this takes. Still nothing published as a skip. This is so the evidence can be looked at from the app rather than from my logs, which is what the decision to publish should rest on. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B