From 7374edc0773988ac3de4d0480a1005d5d1c8cd6d Mon Sep 17 00:00:00 2001 From: Peter Stone Date: Fri, 13 Nov 2026 19:56:19 +0000 Subject: Take the giant Google badge out of the reader "Remove giant Google icon", reported on a West Hawaii Today article. It is the "Add as a preferred source on Google" badge that sits beside the byline. Giant because the page caps it at 120px with its own stylesheet and the reader drops the page's CSS, so a small badge on the site becomes a full-width image above the first paragraph. Matched on the link's address rather than the class, and that is forced rather than preferred: running the vendored Readability.js over the real page shows the wrapper's preferred_source_badge class does not survive extraction at all, so a class rule would match nothing. The anchor does survive, and its address is Google's own and identical for every publisher in the programme -- where the badge's image filename sits under this newspaper's theme directory and is not. So the address is both the only signal left and the one that generalises. A link with words in it is never touched. An article may discuss the programme and link to it in a sentence, and that sentence is the author's. Both cleaners now run from one entry point. There are two places a body is cleaned -- a fresh extraction on the way out, a cached one on the way back in -- and a cached article is cleaned in place rather than discarded, so a cleaner wired into only the fresh path would leave every already-read article carrying litter nothing would ever remove. Peter's report says the article was cached, which is exactly that case. Pinned against the real thing: the reported article, put through the real extractor, loses the badge and keeps its text and byline. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B --- .../feedreader/readability/ArticleLitterTest.kt | 47 +++++++++ .../readability/PreferredSourceBadgeTest.kt | 105 +++++++++++++++++++++ .../west-hawaii-today-usps-extracted.html | 18 ++++ 3 files changed, 170 insertions(+) create mode 100644 app/src/test/java/org/terst/feedreader/readability/ArticleLitterTest.kt create mode 100644 app/src/test/java/org/terst/feedreader/readability/PreferredSourceBadgeTest.kt create mode 100644 app/src/test/resources/west-hawaii-today-usps-extracted.html (limited to 'app/src/test') diff --git a/app/src/test/java/org/terst/feedreader/readability/ArticleLitterTest.kt b/app/src/test/java/org/terst/feedreader/readability/ArticleLitterTest.kt new file mode 100644 index 0000000..121783b --- /dev/null +++ b/app/src/test/java/org/terst/feedreader/readability/ArticleLitterTest.kt @@ -0,0 +1,47 @@ +package org.terst.feedreader.readability + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +/** + * The single entry point both cleaning sites call. + * + * The thing worth testing is that it applies *every* cleaner, because the failure it exists to + * prevent is a cleaner that only runs on one of the two paths. + */ +class ArticleLitterTest { + + private val recirculationWidget = + """
""" + + """

Trending

""" + + private val preferredSourceBadge = + """

""" + + """

""" + + @Test + fun `removes both kinds of litter from one body`() { + val html = "$preferredSourceBadge

Real sentence.

$recirculationWidget" + + val cleaned = ArticleLitter.strip(html)!! + + assertFalse("badge survived: $cleaned", cleaned.contains("preferences/source")) + assertFalse("widget survived: $cleaned", cleaned.contains("Trending")) + assertTrue(cleaned.contains("Real sentence.")) + } + + @Test + fun `leaves a clean article untouched`() { + val html = "

Nothing to clean.

" + + assertEquals(html, ArticleLitter.strip(html)) + } + + @Test + fun `copes with nothing`() { + assertEquals(null, ArticleLitter.strip(null)) + } +} diff --git a/app/src/test/java/org/terst/feedreader/readability/PreferredSourceBadgeTest.kt b/app/src/test/java/org/terst/feedreader/readability/PreferredSourceBadgeTest.kt new file mode 100644 index 0000000..edbb9d0 --- /dev/null +++ b/app/src/test/java/org/terst/feedreader/readability/PreferredSourceBadgeTest.kt @@ -0,0 +1,105 @@ +package org.terst.feedreader.readability + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +/** + * The "Add as a preferred source on Google" badge beside a byline. + * + * Reported live: "remove giant Google icon" on West Hawaii Today. The markup below is what the + * vendored Readability.js actually leaves behind on that article -- the wrapper's class is gone + * by then, which is why the link's address is what gets matched. + */ +class PreferredSourceBadgeTest { + + private val realBadge = + """

""" + + """Add as a preferred source on Google

""" + + @Test + fun `strips the badge that sits beside the byline`() { + val html = "

By A Reporter

$realBadge

The first real sentence.

" + + val cleaned = PreferredSourceBadge.strip(html)!! + + assertFalse("the badge is still there: $cleaned", cleaned.contains("preferred_source_badge")) + assertFalse(cleaned.contains("google.com/preferences/source")) + assertTrue(cleaned.contains("By A Reporter")) + assertTrue(cleaned.contains("The first real sentence.")) + } + + /** It leaves no blank gap where it used to be: the paragraph held nothing else. */ + @Test + fun `takes the emptied paragraph with it`() { + val cleaned = PreferredSourceBadge.strip("

Real.

$realBadge")!! + + assertEquals("

Real.

", cleaned.trim()) + } + + /** + * The safety valve. An article may discuss the programme and link to it in a sentence, and + * that sentence belongs to whoever wrote it. + */ + @Test + fun `leaves a link alone when it has words in it`() { + val html = """

You can """ + + """add us as a preferred source if you like.

""" + + assertEquals(html, PreferredSourceBadge.strip(html)) + } + + @Test + fun `leaves an ordinary image link alone`() { + val html = """

""" + + assertEquals(html, PreferredSourceBadge.strip(html)) + } + + @Test + fun `leaves an article with no badge exactly as it was`() { + val html = "

Nothing to do here.

" + + assertEquals(html, PreferredSourceBadge.strip(html)) + } + + /** + * The whole article, as the app would really hold it. + * + * The fixture is the reported USPS story put through the vendored Readability.js exactly as + * the app does -- the cases above prove the matcher against the badge on its own, this + * proves it meets reality in one piece. Same arrangement as the recirculation widgets, on + * the same newspaper. + */ + @Test + fun `the real extracted article loses the badge and keeps its text`() { + val extracted = checkNotNull( + javaClass.getResourceAsStream("/west-hawaii-today-usps-extracted.html") + ).bufferedReader().use { it.readText() } + assertEquals( + "fixture should carry the reported bug", + 1, + Regex("google_preferred_source_badge").findAll(extracted).count(), + ) + + val cleaned = PreferredSourceBadge.strip(extracted)!! + + assertEquals(0, Regex("google_preferred_source_badge").findAll(cleaned).count()) + assertFalse(cleaned.contains("preferences/source")) + assertTrue(cleaned.contains("The U.S. Postal Service is moving to raise")) + assertTrue(cleaned.contains("Cailley LaPara")) + assertTrue( + "barely any of the article should have gone with the badge", + cleaned.length > extracted.length - 400, + ) + } + + @Test + fun `copes with nothing`() { + assertEquals(null, PreferredSourceBadge.strip(null)) + assertEquals("", PreferredSourceBadge.strip("")) + } +} diff --git a/app/src/test/resources/west-hawaii-today-usps-extracted.html b/app/src/test/resources/west-hawaii-today-usps-extracted.html new file mode 100644 index 0000000..d52ec07 --- /dev/null +++ b/app/src/test/resources/west-hawaii-today-usps-extracted.html @@ -0,0 +1,18 @@ +
+
+

Published 12:05 am Friday, November 13, 2026

+

+ By Cailley LaPara Bloomberg News +

+
+

+ Add as a preferred source on Google + +

+

The U.S. Postal Service is moving to raise and make permanent its fuel surcharge, which the agency introduced for the first time earlier this year.

+

The fuel surcharge will increase to 11% from 8% starting in January, the USPS said in a statement Thursday. Certain domestic shipping services also will increase by an average of 5.9%. That adjustment is in line with the typical annual rate changes that other large parcel carriers have announced for the year ahead.

+

The fuel surcharge hike comes after diesel prices hit record highs due to the war in Iran. The USPS implemented its first-ever fuel related fee in April, soon after the war started and diesel prices jumped. Costs have continued to increase throughout the year.

+

Shares of parcel giants United Parcel Service Inc. fell 1% and FedEx Corp. slipped less than 1% on Thursday after the Postal Service announced the planned price hikes. Both companies are contending with similar cost headwinds and are leaning heavily on pricing to boost revenue.

+

The announcement comes days after the USPS reported annual financial results which showed that the agency lost $6.5 billion in fiscal year 2026, compared with $9 billion the year prior.

+

The USPS faces a dire financial situation due to its massive operational costs and waning mail volume. Postmaster General David Steiner has said the agency could run out of money as soon as February, and has halted discretionary spending and certain retirement contributions to stave off a liquidity crisis. He has also asked Congress for help solving some of the agency’s structural financial issues.

+
\ No newline at end of file -- cgit v1.2.3