From 9210bf463a8818595f7ca42486a787d4f1941d55 Mon Sep 17 00:00:00 2001
From: Peter Stone <thepeterstone@gmail.com>
Date: Wed, 21 Oct 2026 19:07:17 +0000
Subject: Fix stray promo headers, site-URL subscribing, and YouTube playback

Three reported bugs.

**"West Hawaii today articles get a bunch of stray Trending headers."**
The publisher's widget platform injects promo blocks between the real
paragraphs, inside the article's own container, so every structural
signal tells Readability they belong. What survives is a heading
labelling nothing, because the links under it are filled in by a script
that never runs in the reader. Confirmed on 9 of the 12 articles in
that feed, two each. Blocks marked `data-mrf-recirculation` are now
removed -- the attribute says what the block is for in the widget
platform's own words, which is a better signal than the word
"Trending", since an article may legitimately discuss what is trending.
Nothing longer than a label and a few headlines is ever removed. Also
applied to already-cached articles, in place rather than by discarding
them: the article was fine, only the litter has to go.

**"Can't subscribe to heathercoxrichardson.substack.com."** Adding a
feed handed whatever URL it was given straight to the XML parser, so a
site's home page -- the only URL a reader can actually see and copy --
failed on the root element. That page advertises its feed the way sites
have since 2002, in a <link rel="alternate"> in the head. It is now
read, resolved against the page, and fetched. Only what the page
declares: guessing /feed and /rss would add a burst of 404s to every
failed add and eventually subscribe somebody to the wrong thing.

**"YouTube play is misconfigured."** It was, and YouTube said so: the
player dialog loaded the embed as its own top-level document, which
answers with "Video player configuration error -- Error 153". An embed
has to be embedded. It now sits in an iframe on a minimal host page.
Measured in the app's own WebView: top-level, no requests for the video
stream and the error on screen; framed, the stream loads and there is
no error.

**"Long click on YouTube placeholder should open the URL directly."**
It does now. Worth recording why this is not simply the link's href:
for an image inside a link, WebView's hit test reports the image's URL,
so a long-press on the thumbnail arrives as the i.ytimg.com JPEG. The
video id is resolved out of that.

Verified on a device for all four: the Substack site URL adds "Letters
from an American" and fetches 20 articles; a West Hawaii Today article
extracts with no Trending headers and its body intact; the video plays;
and a long-press reaches the YouTube app.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B
---
 .../org/terst/feedreader/data/CachedExtractionTest.kt | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

(limited to 'app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt')

diff --git a/app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt b/app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt
index fc9c32e..442663f 100644
--- a/app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt
+++ b/app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt
@@ -96,4 +96,23 @@ class CachedExtractionTest {
         assertFalse(result.discardedConsentNotice)
         assertFalse(cleared)
     }
+
+    /**
+     * A cached article carrying the publisher's "Trending" promo widgets. Unlike a cookie
+     * notice, the article itself is fine -- only the widgets have to go, and re-extracting
+     * would be a pointless round trip that produced the same body again.
+     */
+    @Test
+    fun `a cached article has its recirculation widgets stripped without being re-extracted`() = runTest {
+        var cleared = false
+        val body = "<p>" + "The actual article body. ".repeat(60) + "</p>"
+        val widget = "<div data-mrf-recirculation=\"IL_x\"><p><h3><span>Trending</span></h3></p></div>"
+
+        val result = loadCachedExtraction(body + widget + body) { cleared = true }
+
+        assertFalse("the stray header survived: ${result.content}", result.content!!.contains("Trending"))
+        assertTrue(result.content!!.contains("The actual article body."))
+        assertFalse("the article was fine; nothing should be thrown away", result.discarded)
+        assertFalse("re-extracting would fetch the same body again", cleared)
+    }
 }
-- 
cgit v1.2.3