From 4fa99073761bbedcdc2bce45a666399bf4b425aa Mon Sep 17 00:00:00 2001 From: Peter Stone Date: Thu, 22 Oct 2026 21:02:29 +0000 Subject: Plan: gift links in the reader, and whether to use kottke's Prometheus Research + plan document only, no code. Covers what Prometheus actually is (member-only beta gift-link search at kottke.org/prometheus, no API, no RSS, no terms page -- all verified at the source, including 404s on every plausible endpoint), where a gift-link action would slot into the existing reader ("Find another source" in ReaderScreen/LinkReaderScreen overflow, the process-wide CookieManager seam BrowserScreen already documents), and what it would cost. Recommends NOT building against Prometheus: it is member-only (Peter isn't a member), one day old, and the only available integration would be scraping a members-only page. Recommends instead subscribing to the public sources it harvests -- mastodon.social's #giftlink RSS and the public Bluesky gift-link feed, both verified reachable unauthenticated -- and waiting for Prometheus to ship a feed or a URL-lookup endpoint before building anything. Two findings worth keeping either way: the app has no paywall state at all (a paywalled page lands as NoContent or a successful teaser extraction, and ReaderExtractor's own comment records why thin-result detection must stay narrow), so there is no reliable automatic trigger for a gift-link offer; and Mastodon tag feeds carry no , which FeedParser leaves as "", so subscribing to #giftlink today would show blank titles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B --- docs/plans/prometheus-gift-links.md | 221 ++++++++++++++++++++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100644 docs/plans/prometheus-gift-links.md (limited to 'docs/plans') diff --git a/docs/plans/prometheus-gift-links.md b/docs/plans/prometheus-gift-links.md new file mode 100644 index 0000000..55dec7c --- /dev/null +++ b/docs/plans/prometheus-gift-links.md @@ -0,0 +1,221 @@ +# Gift links in the reader, and kottke.org's Prometheus + +Researched 2026-10-22, the day after Prometheus was announced. Nothing here is built. + +## Recommendation up front + +**Don't build against Prometheus.** It is a one-day-old beta, member-only, with no API, no +RSS feed, and no documented terms for automated access. Everything an app could do with it +today is HTML-scraping a members-only page through a login session Peter doesn't have. + +**Do subscribe to the sources Prometheus itself harvests.** They are public, free, and +already consumable by this app: + +- `https://mastodon.social/tags/giftlink.rss` — verified live: HTTP 200, + `application/rss+xml`, 20 items, RSS 2.0, items carry the gift URL inside + `<description>` HTML (e.g. an `nytimes.com/...?unlocked_article_code=...` link). +- the Bluesky gift-link feed the announcement links to + (`at://did:plc:o4s55v3tsfph6whswxccpsia/app.bsky.feed.generator/aaaixbb5liqbu`) — verified + readable unauthenticated via `https://public.api.bsky.app/xrpc/app.bsky.feed.getFeed`, + HTTP 200. Not RSS, so it would need an adapter; the Mastodon tag is the cheap one. + +That gets the actual benefit — a stream of paywall-free articles arriving in the reader — +for zero code. It does **not** get the thing Prometheus is actually good for (search by URL: +"is there a gift link for *this* article I just hit a paywall on"), and no public source +provides that. See "Worth it only if" below. + +## What Prometheus is, factually + +Built and run by Jason Kottke, announced 2026-10-21 at +`https://kottke.org/26/10/meet-prometheus-a-gift-link-search-engine`, living at +`https://kottke.org/prometheus/`. + +From the tool's own page (not the article's summary): + +> Prometheus collects gift links to paywalled news articles shared by subscribers — from +> the New York Times, the Washington Post, the Atlantic, and more. Links are gathered from +> Bluesky and Mastodon, updated frequently throughout the day, and you can search by +> keyword & URL and filter by publication. + +> Gift links usually expire about two weeks after they're shared, and expired links are +> removed automatically. Some publications also cap how many times a gift link can be +> opened, so a popular link may stop working sooner. + +Access, stated on both the article and the page: + +> Right now, Prometheus is only available to active kottke.org members while I'm testing +> and refining it — it's definitely a beta tool rn. + +Precisely what requires membership: **the search tool only**. The announcement post is +public and free to read. `https://kottke.org/prometheus/` is also public, but serves a +preview — the description above, plus exactly 4 "today's top links" as fully-formed, +clickable gift URLs — and then "This page is a preview — Prometheus is only available to +current kottke.org members." Search, URL lookup, publication filters and the full list are +behind the member session. Sign-in is Memberful (`kottke.memberful.com/auth/sign_in`). + +Cost of membership, from `https://kottke.org/members`: annual tiers at $30, $60 and $600, +plus $3/mo, $10/mo and $120/yr. Cheapest route in is $30/yr or $3/mo. + +## API reality: there isn't one + +No API is mentioned in the article, on the tool page, or anywhere else found. Verified by +hand rather than assumed: + +- `/prometheus/search?q=`, `/prometheus/feed`, `/prometheus/rss`, `/prometheus/index.xml`, + `/prometheus/api`, `/prometheus/links.json`, `/api/prometheus` — all 404. +- `/prometheus/?q=test` returns the preview page byte-for-byte identical to no query, so + the public page ignores search params entirely. +- The page is server-rendered HTML with two small inline scripts (a theme-before-paint + shim and a clipboard copy handler). There is no `fetch`, no XHR, no JSON endpoint behind + the UI to borrow — so there is no hidden API to discover from the client either. +- No `<link rel="alternate">` feed of any kind on the page. + +What cannot be determined without a membership: whether the member-facing page uses a +different URL or form shape, whether search is a GET with a query param (likely, given the +page has no client-side fetching) or a POST, and whether the member view exposes anything +machine-readable. **This gap is not guessable and the plan deliberately stops here rather +than inventing an endpoint.** Two commenters on the announcement asked for an RSS feed; +Jason has not replied in the thread's 23 comments as of 2026-10-22. + +The markup of the public preview is, for what it's worth, clean and stable-looking to +scrape: `article.gl-item` per link, `span.gl-pub` for the publication, `<time datetime>`, +`h2 > a[href]` for the gift URL, plus a `button.gl-copy[data-url]` carrying the same URL. +That is a 4-item sample, not the index. + +## Where this would fit in the app + +### What the app does with a paywalled article today + +Nothing specific — that's the key finding. There is no paywall state anywhere in the code. +`ExtractionFailure` (`app/src/main/java/org/terst/feedreader/readability/ExtractionFailure.kt`) +enumerates `LoadError`, `HttpError`, `TimedOut`, `ReadabilityError`, `NoContent`, +`BadResult`, `ConsentInterstitial` — no `Paywalled`. A paywalled page therefore lands as +either `NoContent` or, more often, a *successful* extraction of the free teaser. + +And the extractor says explicitly that it must stay that way. In the extraction script +(`ReaderExtractor.kt`, around lines 560-580) there is a deliberate un-hide pass for +elements whose own class/id says `paywall` / `premium-content` / `subscriber-content` / +`metered-content`, and the comment above it records why it is *that* narrow: + +> "Thin normal-pass result" is NOT a reliable paywall signal on its own — plenty of +> ordinary, non-paywalled posts (kottke's linkblog style, for one) are also genuinely just +> short + +So there is no trustworthy automatic trigger for "offer a gift link now". Any gift-link +feature either rides on the existing narrow markup signal (which, when it fires, usually +means the un-hide already *worked* and no gift link is needed) or it is user-initiated. +**User-initiated is the only honest option.** Treat an automatic paywall detector as a +separate, harder prerequisite, not part of this. + +### The seam that already exists + +`ReaderScreen.kt` line ~414 and `LinkReaderScreen.kt` line ~223 both carry an overflow item: + +- **"Find another source"** — URL-encodes the article title into a Google search and hands + it to `onOpenBrowser`, i.e. `BrowserScreen` (a real JS-on mini-browser). +- **"Log in to this site"** — same, with the article's own URL. + +A gift-link action is the same shape as "Find another source", one menu item below it, and +needs no new screen, no new navigation and no new permission. Two variants: + +- *Zero-integration:* open `https://kottke.org/prometheus/` in `BrowserScreen` with the + article's URL already copied to the clipboard. Works whether or not Peter is a member + (member → real search; non-member → the preview page, which is useless for a specific + article). Roughly 15 lines in each of the two menus. +- *Integrated:* query Prometheus for this article's URL, and if a gift link comes back, + re-run `ReaderExtractor.extractResult(context, giftUrl)` instead of `article.link`. The + extractor takes an arbitrary URL already, so the plumbing cost is nearly nil — but this + needs the API that doesn't exist. + +### Authentication would not need new plumbing + +Worth recording because it is the non-obvious part: `BrowserScreen`'s own doc comment +(`BrowserScreen.kt` lines ~40-44) notes that Android's `CookieManager` is a single +process-wide store shared by every WebView in the app, and it turns `setAcceptCookie(true)` +on. So a Memberful login performed once in `BrowserScreen` would be visible to the +extraction WebView in `ReaderExtractor` on later visits to `kottke.org`. A +"log in, then read the member page with the extraction WebView" approach is technically +available today with no new code for auth. It is still not recommended — see below. + +### Where a gift URL would *not* go + +`ArticleEntity` (`data/Entities.kt`) has `link` and `extractedContent` and no field for an +alternate URL. Adding one means `AppDatabase` version 12 plus a migration, and CLAUDE.md's +gate #8 requires a migration test for that — of which this repo currently has none for any +of its 11 versions. Given gift links expire in ~2 weeks and can be burned out by open-count +caps, persisting one is mostly a way to cache a dead URL. **Keep any gift URL ephemeral** +(screen state only) and let the existing `articles.extractedContent` cache hold the +*content*, which does not expire. That keeps the whole feature out of the schema. + +## Worth it only if + +One of these becomes true: + +1. **Prometheus ships an RSS feed.** Then the work is "add a feed" — possibly literally + zero code — and it lands in the stream Peter already reads. Two members have already + asked. This is the outcome to wait for. +2. **Prometheus ships a URL-lookup endpoint** (`is there a gift link for this URL?`), and + Peter is a member. Then the integrated variant above is a genuinely good feature: hit a + paywall, tap one menu item, read the article. Small diff, real payoff. +3. Peter becomes a member for his own reasons and wants the zero-integration menu item as a + convenience. That one is cheap and defensible on its own. + +Not worth it: scraping the member page. It is a beta the author says he is actively +refining, so the selectors are the least stable thing about it; the benefit accrues to one +user; and it would mean driving a members-only page with the JS-off extraction WebView +rather than a browser. If this ever looks worth doing, the better move is to email Jason and +ask for a feed or an endpoint — he is one reachable person, not a platform. + +## Blockers and open questions + +- **Membership.** Peter is not a member ("I'm not currently a kottke member so it's not + immediately useful"), so nothing member-gated can even be evaluated, let alone built + against. Not signed up for while researching this, by instruction. +- **Unknown member-side shape.** Search URL, parameter names, result markup, pagination, + rate limits: all unobservable from outside. Stated as unknown rather than guessed. +- **Beta churn.** Announced yesterday. Anything built against its HTML today is a bet on + markup that is explicitly still moving. +- **Link lifetime.** ~2-week expiry plus per-publication open caps means a gift URL is a + perishable; any design that stores or shares one has to tolerate it being dead on arrival. +- **No paywall signal in the app** (see above). Automatic triggering is its own project. +- **Mastodon tag items have no `<title>`.** Verified in the live feed: items carry only + `guid`/`link`/`pubDate`/`description`. `FeedParser.parseRss` leaves `itemTitle` as `""` + in that case (it initialises to `""` and only overwrites from a `<title>` element), so + subscribing to `giftlink.rss` today would produce article rows with blank titles. That is + the one code change actually worth considering out of this whole exercise, and it is not + Prometheus-specific: a title fallback (first sentence of content, or author + date) for + any title-less feed. Needs a test first, per CLAUDE.md. +- **Mastodon tag feeds are single-instance.** `mastodon.social`'s `#giftlink` tag feed sees + the tag as that instance's federated view, not the whole network — it is a sample, not the + firehose. Prometheus aggregating several sources is a real advantage over subscribing to + one tag. + +## Legal/terms notes + +- kottke.org publishes no terms-of-use page (`/terms` and `/tos` both 404), so there is no + document that permits or forbids automated querying. +- `robots.txt` disallows `/cgi-bin`, `/stats` and `/tools/` for all user agents, blocks + `CCBot` and `anthropic-ai`, and explicitly allows `GPTBot`, `OAI-SearchBot`, `ClaudeBot` + and `Google-Extended`. `/prometheus/` is not disallowed. That governs crawlers; a + user-initiated in-app fetch is a different thing, and nothing public addresses it either + way. +- The only stated restriction is the membership gate itself, which is enforced by session. + Automating a logged-in member session against a members-only beta is not addressed + anywhere, which is a reason to ask rather than a reason to assume. +- Separately: the gift links themselves are the publishers' own sharing mechanism, used as + intended. The publisher-side open-count caps are the enforcement, and nothing here would + try to work around them. + +## What would need device verification, if any of this is ever built + +- That `ReaderExtractor` extracts a real gift URL cleanly — i.e. that a NYT/WaPo/Atlantic + gift link's query token survives into a successful Readability pass with the full body, + not the teaser. Untested; the extractor loads the URL as given (`ReaderExtractor.kt` line + ~689), so there is no known reason it would strip the token, but "no known reason" is not + a measurement. +- That opening a gift link once from the app doesn't burn the open-count in a way that + makes a second read fail (the extraction WebView plus a later "Open original" would be + two opens). +- Whether a Memberful session set in `BrowserScreen` really does reach the extraction + WebView for `kottke.org` — the `CookieManager` claim is from the code's own comment and + the platform's documented behaviour, not from an observed login. -- cgit v1.2.3