Take the giant Google badge out of the readerHEADmaster

"Remove giant Google icon", reported on a West Hawaii Today article. It is the
"Add as a preferred source on Google" badge that sits beside the byline. Giant
because the page caps it at 120px with its own stylesheet and the reader drops
the page's CSS, so a small badge on the site becomes a full-width image above
the first paragraph.

Matched on the link's address rather than the class, and that is forced rather
than preferred: running the vendored Readability.js over the real page shows
the wrapper's preferred_source_badge class does not survive extraction at all,
so a class rule would match nothing. The anchor does survive, and its address
is Google's own and identical for every publisher in the programme -- where the
badge's image filename sits under this newspaper's theme directory and is not.
So the address is both the only signal left and the one that generalises.

A link with words in it is never touched. An article may discuss the programme
and link to it in a sentence, and that sentence is the author's.

Both cleaners now run from one entry point. There are two places a body is
cleaned -- a fresh extraction on the way out, a cached one on the way back in --
and a cached article is cleaned in place rather than discarded, so a cleaner
wired into only the fresh path would leave every already-read article carrying
litter nothing would ever remove. Peter's report says the article was cached,
which is exactly that case.

Pinned against the real thing: the reported article, put through the real
extractor, loses the badge and keeps its text and byline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

2 files changed, 152 insertions, 0 deletions

diff --git a/app/src/test/java/org/terst/feedreader/readability/ArticleLitterTest.kt b/app/src/test/java/org/terst/feedreader/readability/ArticleLitterTest.kt
new file mode 100644
index 0000000..121783b
--- /dev/null
+++ b/app/src/test/java/org/terst/feedreader/readability/ArticleLitterTest.kt
@@ -0,0 +1,47 @@
+package org.terst.feedreader.readability
+
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertFalse
+import org.junit.Assert.assertTrue
+import org.junit.Test
+
+/**
+ * The single entry point both cleaning sites call.
+ *
+ * The thing worth testing is that it applies *every* cleaner, because the failure it exists to
+ * prevent is a cleaner that only runs on one of the two paths.
+ */
+class ArticleLitterTest {
+
+    private val recirculationWidget =
+        """<div data-mrf-recirculation="IL_x" data-testid="text-html">""" +
+            """<p><h3><span>Trending</span></h3></p></div>"""
+
+    private val preferredSourceBadge =
+        """<p><a href="https://www.google.com/preferences/source?q=westhawaiitoday.com">""" +
+            """<img src="https://example.com/google_preferred_source_badge_light_en.png" """ +
+            """alt="Add as a preferred source on Google"></a></p>"""
+
+    @Test
+    fun `removes both kinds of litter from one body`() {
+        val html = "$preferredSourceBadge<p>Real sentence.</p>$recirculationWidget"
+
+        val cleaned = ArticleLitter.strip(html)!!
+
+        assertFalse("badge survived: $cleaned", cleaned.contains("preferences/source"))
+        assertFalse("widget survived: $cleaned", cleaned.contains("Trending"))
+        assertTrue(cleaned.contains("Real sentence."))
+    }
+
+    @Test
+    fun `leaves a clean article untouched`() {
+        val html = "<p>Nothing to clean.</p>"
+
+        assertEquals(html, ArticleLitter.strip(html))
+    }
+
+    @Test
+    fun `copes with nothing`() {
+        assertEquals(null, ArticleLitter.strip(null))
+    }
+}
diff --git a/app/src/test/java/org/terst/feedreader/readability/PreferredSourceBadgeTest.kt b/app/src/test/java/org/terst/feedreader/readability/PreferredSourceBadgeTest.kt
new file mode 100644
index 0000000..edbb9d0
--- /dev/null
+++ b/app/src/test/java/org/terst/feedreader/readability/PreferredSourceBadgeTest.kt
@@ -0,0 +1,105 @@
+package org.terst.feedreader.readability
+
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertFalse
+import org.junit.Assert.assertTrue
+import org.junit.Test
+
+/**
+ * The "Add as a preferred source on Google" badge beside a byline.
+ *
+ * Reported live: "remove giant Google icon" on West Hawaii Today. The markup below is what the
+ * vendored Readability.js actually leaves behind on that article -- the wrapper's class is gone
+ * by then, which is why the link's address is what gets matched.
+ */
+class PreferredSourceBadgeTest {
+
+    private val realBadge =
+        """<p><a href="https://www.google.com/preferences/source?q=westhawaiitoday.com" """ +
+            """target="_blank" rel="noopener"> """ +
+            """<img src="https://www.westhawaiitoday.com/wp-content/themes/2024-cmg/media/img/""" +
+            """google_preferred_source_badge_light_en.png" alt="Add as a preferred source on Google"> </a> </p>"""
+
+    @Test
+    fun `strips the badge that sits beside the byline`() {
+        val html = "<div><p>By A Reporter</p>$realBadge</div><p>The first real sentence.</p>"
+
+        val cleaned = PreferredSourceBadge.strip(html)!!
+
+        assertFalse("the badge is still there: $cleaned", cleaned.contains("preferred_source_badge"))
+        assertFalse(cleaned.contains("google.com/preferences/source"))
+        assertTrue(cleaned.contains("By A Reporter"))
+        assertTrue(cleaned.contains("The first real sentence."))
+    }
+
+    /** It leaves no blank gap where it used to be: the paragraph held nothing else. */
+    @Test
+    fun `takes the emptied paragraph with it`() {
+        val cleaned = PreferredSourceBadge.strip("<p>Real.</p>$realBadge")!!
+
+        assertEquals("<p>Real.</p>", cleaned.trim())
+    }
+
+    /**
+     * The safety valve. An article may discuss the programme and link to it in a sentence, and
+     * that sentence belongs to whoever wrote it.
+     */
+    @Test
+    fun `leaves a link alone when it has words in it`() {
+        val html = """<p>You can <a href="https://www.google.com/preferences/source?q=x">""" +
+            """add us as a preferred source</a> if you like.</p>"""
+
+        assertEquals(html, PreferredSourceBadge.strip(html))
+    }
+
+    @Test
+    fun `leaves an ordinary image link alone`() {
+        val html = """<p><a href="https://example.com/photo"><img src="https://example.com/p.jpg"></a></p>"""
+
+        assertEquals(html, PreferredSourceBadge.strip(html))
+    }
+
+    @Test
+    fun `leaves an article with no badge exactly as it was`() {
+        val html = "<p>Nothing to do here.</p>"
+
+        assertEquals(html, PreferredSourceBadge.strip(html))
+    }
+
+    /**
+     * The whole article, as the app would really hold it.
+     *
+     * The fixture is the reported USPS story put through the vendored Readability.js exactly as
+     * the app does -- the cases above prove the matcher against the badge on its own, this
+     * proves it meets reality in one piece. Same arrangement as the recirculation widgets, on
+     * the same newspaper.
+     */
+    @Test
+    fun `the real extracted article loses the badge and keeps its text`() {
+        val extracted = checkNotNull(
+            javaClass.getResourceAsStream("/west-hawaii-today-usps-extracted.html")
+        ).bufferedReader().use { it.readText() }
+        assertEquals(
+            "fixture should carry the reported bug",
+            1,
+            Regex("google_preferred_source_badge").findAll(extracted).count(),
+        )
+
+        val cleaned = PreferredSourceBadge.strip(extracted)!!
+
+        assertEquals(0, Regex("google_preferred_source_badge").findAll(cleaned).count())
+        assertFalse(cleaned.contains("preferences/source"))
+        assertTrue(cleaned.contains("The U.S. Postal Service is moving to raise"))
+        assertTrue(cleaned.contains("Cailley LaPara"))
+        assertTrue(
+            "barely any of the article should have gone with the badge",
+            cleaned.length > extracted.length - 400,
+        )
+    }
+
+    @Test
+    fun `copes with nothing`() {
+        assertEquals(null, PreferredSourceBadge.strip(null))
+        assertEquals("", PreferredSourceBadge.strip(""))
+    }
+}