|
The machine that hosts a feed and the machine that downloads the audio do
not have to be the same, and usually shouldn't be: a datacentre IP is
treated very differently from a home connection by the larger video sites.
Downloading where the person actually is sidesteps that entirely, and means
the server needs no credentials for or route to the source.
- push/ingest: `podkit push` uploads a local library to a remote
`serve --ingest`, asking for a manifest first so only the difference is
sent. Uploads land via a temp file and rename, so an interrupted transfer
cannot be picked up as a short episode. Ingest is off by default and
refuses to start without a token.
- get: downloads a direct audio URL or every enclosure in an RSS feed,
with no extractor involved. Numbers RSS items in reverse so a mirrored
series plays in order rather than backwards.
Two bugs found by running it rather than by testing it:
- A tokened server produced enclosure URLs with no token. The feed loaded,
listed every episode, and all of them 404'd in a real client — the whole
point of the tool, broken in the default configuration. Feeds now carry
the token; regression test asserts enclosures from a tokened feed
actually resolve against a tokened server.
- Ingest ran filepath.Base before validating, so "../evil.mp3" was stored
as "evil.mp3" — contained, but under a name the client never asked for,
leaving push and the server disagreeing about the library. Names are now
validated raw and rejected.
Also: identify with a real user-agent (libsyn 403s Go's default), write
feed.xml via rename so a polling client never reads a truncated feed, and
match ingested file permissions to downloaded ones.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B
|
|
One Go binary with fetch / tag / feed / serve / all. The stages run separately
on purpose: the download is the slow part and should not be repeated while
adjusting titles or the base URL.
Built from a shell + Python + Go sketch. The pieces that needed fixing on the
way, all of which produce a feed that loads wrong or not at all:
- Enclosure URLs were unescaped. Track titles routinely contain spaces and
ampersands, which make both the URL and the XML invalid — the likeliest way
for a feed to fail to load entirely. Only the final path segment is escaped;
escaping the base would mangle scheme and host.
- Ordering came from filename sort, which is correct only while zero-padding is
applied consistently. One unpadded file puts lecture 10 after lecture 1.
Order now comes from the parsed track number.
- No pubDate and no itunes:type. Podcast clients default to newest-first, which
starts a lecture course at the final lecture. Items now carry distinct
ascending dates and the feed is marked serial.
- Missing channel description and link, which several clients reject outright.
- A struct field with no xml tag emitted a stray <EnclosureURL> element.
- guid had no isPermaLink="false", so a reader may treat it as a fetchable URL.
- Errors from ReadDir, Info and Marshal were discarded, so a missing directory
produced an empty feed that looked like a working setup with nothing in it.
- os.Args indexed without a length check.
Beyond the sketch: a download archive so an interrupted series resumes, an
up-front check for yt-dlp and ffmpeg before the long download rather than a
post-processing failure after the bytes are on disk, non-audio files excluded
by extension so cover art and .part files cannot become episodes, and a serve
command with token gating that 404s rather than 401s.
Found by the first live run against a LibriVox collection: sources whose own
titles begin with a track number came out as "01 - 01 - April.mp3", because the
output template prepends the playlist index regardless. Collapsed, but only
when the two numbers agree — "01 - 1964 - Lecture" is a year and "05 - 12 Angry
Men" is a title, and neither is a repeat.
47 tests. Verified end to end against public-domain audio: downloaded,
converted, tagged, served, and the delivered file is a playable 128kbps MP3
with correct ID3 title/artist/album/track frames.
|
