summaryrefslogtreecommitdiff
path: root/app/src/test/java/org/terst/feedreader/data
diff options
context:
space:
mode:
authorPeter Stone <thepeterstone@gmail.com>2026-10-21 19:07:17 +0000
committerPeter Stone <thepeterstone@gmail.com>2026-10-21 19:07:17 +0000
commit9210bf463a8818595f7ca42486a787d4f1941d55 (patch)
tree3a38cf5d71a20393a33eef5a19b1a97554055e75 /app/src/test/java/org/terst/feedreader/data
parente50f4fbbca21b85648f6c4045cbd50f672d315cd (diff)
Fix stray promo headers, site-URL subscribing, and YouTube playback
Three reported bugs. **"West Hawaii today articles get a bunch of stray Trending headers."** The publisher's widget platform injects promo blocks between the real paragraphs, inside the article's own container, so every structural signal tells Readability they belong. What survives is a heading labelling nothing, because the links under it are filled in by a script that never runs in the reader. Confirmed on 9 of the 12 articles in that feed, two each. Blocks marked `data-mrf-recirculation` are now removed -- the attribute says what the block is for in the widget platform's own words, which is a better signal than the word "Trending", since an article may legitimately discuss what is trending. Nothing longer than a label and a few headlines is ever removed. Also applied to already-cached articles, in place rather than by discarding them: the article was fine, only the litter has to go. **"Can't subscribe to heathercoxrichardson.substack.com."** Adding a feed handed whatever URL it was given straight to the XML parser, so a site's home page -- the only URL a reader can actually see and copy -- failed on the root element. That page advertises its feed the way sites have since 2002, in a <link rel="alternate"> in the head. It is now read, resolved against the page, and fetched. Only what the page declares: guessing /feed and /rss would add a burst of 404s to every failed add and eventually subscribe somebody to the wrong thing. **"YouTube play is misconfigured."** It was, and YouTube said so: the player dialog loaded the embed as its own top-level document, which answers with "Video player configuration error -- Error 153". An embed has to be embedded. It now sits in an iframe on a minimal host page. Measured in the app's own WebView: top-level, no requests for the video stream and the error on screen; framed, the stream loads and there is no error. **"Long click on YouTube placeholder should open the URL directly."** It does now. Worth recording why this is not simply the link's href: for an image inside a link, WebView's hit test reports the image's URL, so a long-press on the thumbnail arrives as the i.ytimg.com JPEG. The video id is resolved out of that. Verified on a device for all four: the Substack site URL adds "Letters from an American" and fetches 20 articles; a West Hawaii Today article extracts with no Trending headers and its body intact; the video plays; and a long-press reaches the YouTube app. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B
Diffstat (limited to 'app/src/test/java/org/terst/feedreader/data')
-rw-r--r--app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt19
-rw-r--r--app/src/test/java/org/terst/feedreader/data/FeedDiscoveryTest.kt93
2 files changed, 112 insertions, 0 deletions
diff --git a/app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt b/app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt
index fc9c32e..442663f 100644
--- a/app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt
+++ b/app/src/test/java/org/terst/feedreader/data/CachedExtractionTest.kt
@@ -96,4 +96,23 @@ class CachedExtractionTest {
assertFalse(result.discardedConsentNotice)
assertFalse(cleared)
}
+
+ /**
+ * A cached article carrying the publisher's "Trending" promo widgets. Unlike a cookie
+ * notice, the article itself is fine -- only the widgets have to go, and re-extracting
+ * would be a pointless round trip that produced the same body again.
+ */
+ @Test
+ fun `a cached article has its recirculation widgets stripped without being re-extracted`() = runTest {
+ var cleared = false
+ val body = "<p>" + "The actual article body. ".repeat(60) + "</p>"
+ val widget = "<div data-mrf-recirculation=\"IL_x\"><p><h3><span>Trending</span></h3></p></div>"
+
+ val result = loadCachedExtraction(body + widget + body) { cleared = true }
+
+ assertFalse("the stray header survived: ${result.content}", result.content!!.contains("Trending"))
+ assertTrue(result.content!!.contains("The actual article body."))
+ assertFalse("the article was fine; nothing should be thrown away", result.discarded)
+ assertFalse("re-extracting would fetch the same body again", cleared)
+ }
}
diff --git a/app/src/test/java/org/terst/feedreader/data/FeedDiscoveryTest.kt b/app/src/test/java/org/terst/feedreader/data/FeedDiscoveryTest.kt
new file mode 100644
index 0000000..eee238a
--- /dev/null
+++ b/app/src/test/java/org/terst/feedreader/data/FeedDiscoveryTest.kt
@@ -0,0 +1,93 @@
+package org.terst.feedreader.data
+
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertNull
+import org.junit.Test
+
+/**
+ * Finding the feed when what you were given is a site.
+ *
+ * Reported live: "Can't subscribe to https://heathercoxrichardson.substack.com/". Pasting a
+ * site's home page was the only thing to hand -- a site does not advertise its feed URL
+ * anywhere a reader can see, it advertises it in a <link rel="alternate"> that every other
+ * reader looks for and this one did not.
+ */
+class FeedDiscoveryTest {
+
+ /** Verbatim from the reported page, 2026-10-21. */
+ private val substackHead =
+ """<!doctype html><html><head><title>Letters from an American</title>""" +
+ """<link rel="alternate" type="application/rss+xml" href="/feed" """ +
+ """title="Letters from an American"/></head><body>...</body></html>"""
+
+ @Test
+ fun `finds the feed a Substack home page advertises, resolved against the site`() {
+ assertEquals(
+ "https://heathercoxrichardson.substack.com/feed",
+ FeedDiscovery.findFeedUrl(substackHead, "https://heathercoxrichardson.substack.com/")
+ )
+ }
+
+ @Test
+ fun `finds an Atom feed too`() {
+ val html = """<link rel="alternate" type="application/atom+xml" href="/atom.xml">"""
+
+ assertEquals("https://example.com/atom.xml", FeedDiscovery.findFeedUrl(html, "https://example.com/"))
+ }
+
+ @Test
+ fun `keeps an absolute href as it stands`() {
+ val html = """<link rel="alternate" type="application/rss+xml" href="https://feeds.example.net/x.xml">"""
+
+ assertEquals("https://feeds.example.net/x.xml", FeedDiscovery.findFeedUrl(html, "https://example.com/blog/"))
+ }
+
+ @Test
+ fun `resolves a relative href against the page, not the site root`() {
+ val html = """<link rel="alternate" type="application/rss+xml" href="feed.xml">"""
+
+ assertEquals("https://example.com/blog/feed.xml", FeedDiscovery.findFeedUrl(html, "https://example.com/blog/"))
+ }
+
+ @Test
+ fun `copes with the attributes in any order and single quotes`() {
+ val html = """<link type='application/rss+xml' href='/rss' rel='alternate'>"""
+
+ assertEquals("https://example.com/rss", FeedDiscovery.findFeedUrl(html, "https://example.com"))
+ }
+
+ @Test
+ fun `prefers the first feed when a page advertises several`() {
+ val html =
+ """<link rel="alternate" type="application/rss+xml" href="/feed" title="Posts">""" +
+ """<link rel="alternate" type="application/rss+xml" href="/comments/feed" title="Comments">"""
+
+ assertEquals("https://example.com/feed", FeedDiscovery.findFeedUrl(html, "https://example.com/"))
+ }
+
+ @Test
+ fun `ignores an alternate link that is not a feed`() {
+ val html =
+ """<link rel="alternate" hreflang="fr" href="/fr/">""" +
+ """<link rel="alternate" type="application/json" href="/feed.json">"""
+
+ assertNull(FeedDiscovery.findFeedUrl(html, "https://example.com/"))
+ }
+
+ @Test
+ fun `a page with no feed at all reports nothing rather than guessing a path`() {
+ assertNull(FeedDiscovery.findFeedUrl("<html><head></head><body>hi</body></html>", "https://example.com/"))
+ }
+
+ @Test
+ fun `an http feed link is upgraded, since the app cannot fetch cleartext`() {
+ val html = """<link rel="alternate" type="application/rss+xml" href="http://example.com/feed">"""
+
+ assertEquals("https://example.com/feed", FeedDiscovery.findFeedUrl(html, "https://example.com/"))
+ }
+
+ @Test
+ fun `xml declaration means this was already a feed, so there is nothing to discover`() {
+ assertNull(FeedDiscovery.findFeedUrl("<?xml version=\"1.0\"?><rss><channel></channel></rss>", "https://x/f"))
+ }
+}