From 9210bf463a8818595f7ca42486a787d4f1941d55 Mon Sep 17 00:00:00 2001
From: Peter Stone <thepeterstone@gmail.com>
Date: Wed, 21 Oct 2026 19:07:17 +0000
Subject: Fix stray promo headers, site-URL subscribing, and YouTube playback

Three reported bugs.

**"West Hawaii today articles get a bunch of stray Trending headers."**
The publisher's widget platform injects promo blocks between the real
paragraphs, inside the article's own container, so every structural
signal tells Readability they belong. What survives is a heading
labelling nothing, because the links under it are filled in by a script
that never runs in the reader. Confirmed on 9 of the 12 articles in
that feed, two each. Blocks marked `data-mrf-recirculation` are now
removed -- the attribute says what the block is for in the widget
platform's own words, which is a better signal than the word
"Trending", since an article may legitimately discuss what is trending.
Nothing longer than a label and a few headlines is ever removed. Also
applied to already-cached articles, in place rather than by discarding
them: the article was fine, only the litter has to go.

**"Can't subscribe to heathercoxrichardson.substack.com."** Adding a
feed handed whatever URL it was given straight to the XML parser, so a
site's home page -- the only URL a reader can actually see and copy --
failed on the root element. That page advertises its feed the way sites
have since 2002, in a <link rel="alternate"> in the head. It is now
read, resolved against the page, and fetched. Only what the page
declares: guessing /feed and /rss would add a burst of 404s to every
failed add and eventually subscribe somebody to the wrong thing.

**"YouTube play is misconfigured."** It was, and YouTube said so: the
player dialog loaded the embed as its own top-level document, which
answers with "Video player configuration error -- Error 153". An embed
has to be embedded. It now sits in an iframe on a minimal host page.
Measured in the app's own WebView: top-level, no requests for the video
stream and the error on screen; framed, the stream loads and there is
no error.

**"Long click on YouTube placeholder should open the URL directly."**
It does now. Worth recording why this is not simply the link's href:
for an image inside a link, WebView's hit test reports the image's URL,
so a long-press on the thumbnail arrives as the i.ytimg.com JPEG. The
video id is resolved out of that.

Verified on a device for all four: the Substack site URL adds "Letters
from an American" and fetches 20 articles; a West Hawaii Today article
extracts with no Trending headers and its body intact; the video plays;
and a long-press reaches the YouTube app.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B
---
 .../java/org/terst/feedreader/ui/YouTubeLinks.kt   | 127 +++++++++++++++++++++
 1 file changed, 127 insertions(+)
 create mode 100644 app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt

(limited to 'app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt')

diff --git a/app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt b/app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt
new file mode 100644
index 0000000..e2b3b62
--- /dev/null
+++ b/app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt
@@ -0,0 +1,127 @@
+package org.terst.feedreader.ui
+
+/**
+ * Recognising a YouTube placeholder link, and building a page its embed will actually play in.
+ *
+ * Split out of ReaderWebView for the usual reason: this is string work with no WebView, no
+ * gestures and no Compose in it, so it can be exercised without a device -- which matters here
+ * because the bug it fixes is invisible until a video is tapped.
+ *
+ * **Why the embed needs a page at all.** The player dialog used to call
+ * `loadUrl("https://www.youtube-nocookie.com/embed/<id>")`, loading the embed as the WebView's
+ * own top-level document. YouTube refuses that: the player comes up showing "Video player
+ * configuration error -- Error 153", which is reported as "YouTube play is misconfigured".
+ * Confirmed by loading the same URL three ways in a real browser on an Android user agent:
+ *
+ * | how the embed is loaded                  | result                      |
+ * |------------------------------------------|-----------------------------|
+ * | top-level, as the app did                | Error 153                   |
+ * | top-level with `&origin=`                | Error 153                   |
+ * | in an `<iframe>` on an ordinary https page | plays                     |
+ *
+ * An embed is meant to be embedded. Given a host page it behaves exactly as it does on any
+ * site that includes a video, so that is what [youTubeEmbedPageHtml] provides.
+ */
+
+/** Video id characters, per YouTube: 11 of these and nothing else. */
+private val VIDEO_ID_CHARS = Regex("[^A-Za-z0-9_-]")
+
+/** Hosts whose links are YouTube's. Matched whole, so `notyoutube.com` is not one of them. */
+private val YOUTUBE_HOSTS = setOf("youtube.com", "www.youtube.com", "m.youtube.com", "youtu.be", "www.youtu.be")
+
+/** Thumbnail hosts. YouTube serves these from i.ytimg.com and numbered variants of it. */
+private val THUMBNAIL_HOST = Regex("^i[0-9]*\\.ytimg\\.com$")
+
+/** `/vi/<id>/...` or `/vi_webp/<id>/...` -- the id is the first segment after the vi folder. */
+private val THUMBNAIL_PATH = Regex("^/vi(?:_webp)?/([A-Za-z0-9_-]{11})/")
+
+/** `?v=<id>` anywhere in the query. */
+private val WATCH_QUERY = Regex("[?&]v=([A-Za-z0-9_-]{11})(?:&|$)")
+
+/** `/<id>` as the whole path, which is what youtu.be links are. */
+private val SHORT_PATH = Regex("^/([A-Za-z0-9_-]{11})/?$")
+
+/**
+ * The origin the embed is hosted on.
+ *
+ * Deliberately not a youtube.com domain: the embed checks that it is framed by a page other
+ * than its own, which is the condition the top-level load failed. Nothing is ever fetched from
+ * this address -- it is the base URL for `loadDataWithBaseURL`, so it only has to be a
+ * well-formed https origin for the document to have one.
+ */
+const val YOUTUBE_EMBED_BASE_URL = "https://reader.feedreader.invalid/"
+
+/**
+ * The video a long-press reported, as a watch URL, or null when the long-press was not on one.
+ *
+ * **What a long-press actually reports.** Not the link's href. The reader's video placeholder is
+ * `<a href="https://www.youtube.com/watch?v=ID"><img src="https://i.ytimg.com/vi/ID/hq720.jpg"></a>`,
+ * and for an image inside a link WebView's hit test gives SRC_IMAGE_ANCHOR_TYPE with the
+ * *image's* URL in `getExtra()`, not the anchor's. Confirmed on the device: long-pressing the
+ * thumbnail opened the ordinary link menu showing the i.ytimg.com JPEG. So the thumbnail URL is
+ * the shape that has to be recognised, and it carries the video id in its path.
+ *
+ * Watch and youtu.be links are recognised too, for wherever one does arrive -- a plain text link
+ * to a video in an article, which is not wrapped in an image.
+ *
+ * Anything on YouTube that is not one specific video -- a channel, a playlist -- returns null and
+ * gets the ordinary actions menu, since "open this directly" has no single video to mean.
+ */
+fun youTubeTargetForLongPress(url: String): String? {
+    val host = hostOf(url) ?: return null
+    val path = pathOf(url)
+    val id = when {
+        THUMBNAIL_HOST.matches(host) -> THUMBNAIL_PATH.find(path)?.groupValues?.get(1)
+        host !in YOUTUBE_HOSTS -> null
+        host.endsWith("youtu.be") -> SHORT_PATH.find(path)?.groupValues?.get(1)
+        else -> WATCH_QUERY.find(url)?.groupValues?.get(1)
+    }
+    return id?.let { youTubeWatchUrl(it) }
+}
+
+/** Where a video lives, in the form a phone hands to the YouTube app. */
+fun youTubeWatchUrl(videoId: String): String = "https://www.youtube.com/watch?v=${scrub(videoId)}"
+
+/**
+ * A minimal page holding nothing but the player, filling it.
+ *
+ * `allow="autoplay"` is required for the autoplay in the src to be honoured inside an iframe --
+ * without it the embed loads and then sits there, which would look like a different bug with
+ * the same symptom.
+ */
+fun youTubeEmbedPageHtml(videoId: String): String {
+    val id = scrub(videoId)
+    return """<!doctype html><html><head><meta name="viewport" content="width=device-width,initial-scale=1">""" +
+        """<style>html,body{margin:0;padding:0;height:100%;background:#000;overflow:hidden}""" +
+        """iframe{display:block;border:0;width:100%;height:100%}</style></head><body>""" +
+        """<iframe src="https://www.youtube-nocookie.com/embed/$id?autoplay=1&playsinline=1&rel=0" """ +
+        """allow="autoplay; encrypted-media; picture-in-picture; fullscreen" allowfullscreen></iframe>""" +
+        """</body></html>"""
+}
+
+/**
+ * The id with anything outside YouTube's own alphabet removed.
+ *
+ * The ids reaching here are already matched against `[a-zA-Z0-9_-]{11}` upstream, so this
+ * changes nothing today. It exists because the value is spliced straight into an HTML
+ * attribute, and that should be safe on its own terms rather than because of a check in a
+ * different file that a later edit might move.
+ */
+private fun scrub(videoId: String): String = VIDEO_ID_CHARS.replace(videoId, "")
+
+/** The path of [url], or "/" when it has none. */
+private fun pathOf(url: String): String {
+    val afterScheme = url.substringAfter("://", missingDelimiterValue = "")
+    val afterAuthority = afterScheme.substringAfter('/', missingDelimiterValue = "")
+    val path = "/" + afterAuthority.substringBefore('?').substringBefore('#')
+    return path
+}
+
+/** The host of [url], lowercased, or null when it is not a URL with one. */
+private fun hostOf(url: String): String? {
+    val afterScheme = url.substringAfter("://", missingDelimiterValue = "")
+    if (afterScheme.isEmpty()) return null
+    val authority = afterScheme.substringBefore('/').substringBefore('?').substringBefore('#')
+    val host = authority.substringAfterLast('@').substringBefore(':')
+    return host.lowercase().ifEmpty { null }
+}
-- 
cgit v1.2.3