Stop reader mode passing a site's cookie notice off as the article

Reported live: reader mode on a New Republic feature showed nothing but
the cookie disclaimer, and had cached it.

Reproduced outside the app by running ReaderExtractor's own injected
script (Readability.js and all) against the real page. With the article
body present, extraction is perfect. With the body absent from the DOM at
the moment the script runs, Readability's first pass -- which discards
"unlikely candidates", and the Didomi banner's class is
didomi-notice-banner -- comes back under its own 500-character threshold,
so it retries with that stripping switched off. The banner's single
paragraph is 510 characters, i.e. just over the threshold, so it comes
back as the article with the right title attached.

Nothing on the Kotlin side looked at that. Any non-blank content counted
as ExtractionResult.Success, was written to articles.extractedContent,
and was never reconsidered -- so one bad load became a permanent cookie
disclaimer, which is the "extractedContent cached: true" in the report.

ConsentNotice recognises such a result: a hard plain-text length cap
first (nothing long is ever rejected, however much consent boilerplate it
quotes), then either a known CMP container id -- Readability strips
classes but keeps ids -- or two or more distinct consent phrases.

The accept/retry/give-up decision moves out of the WebView callback into
decideEvalOutcome so it can be tested without a device. A consent notice
is now "not ready yet", which routes it into the retry the extractor
already does for an empty result; if it survives all six attempts that is
what the page serves, and it fails as ExtractionFailure.ConsentInterstitial
rather than being accepted.

A fixed extractor cannot repair what is already on disk, so
loadCachedExtraction drops a cached notice on read and re-extracts, in
both ReaderScreen and LinkReaderScreen. No schema change: the repair
happens when the article is next opened. Reader mode is still never
turned on heuristically -- replacing a discarded notice does not flip it.

The bug report now also carries the last extraction error, without which
a report about bad reader content cannot say whether extraction failed.

Tests: 59 -> 78.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

2 files changed, 28 insertions, 5 deletions

diff --git a/app/src/main/java/org/terst/feedreader/ui/LinkReaderScreen.kt b/app/src/main/java/org/terst/feedreader/ui/LinkReaderScreen.kt
index 4ed0722..72f6c1f 100644
--- a/app/src/main/java/org/terst/feedreader/ui/LinkReaderScreen.kt
+++ b/app/src/main/java/org/terst/feedreader/ui/LinkReaderScreen.kt
@@ -51,6 +51,7 @@ import org.terst.feedreader.data.DEFAULT_READER_FONT_SIZE_PX
 import org.terst.feedreader.data.Settings
 import org.terst.feedreader.data.ensureStandaloneArticle
 import org.terst.feedreader.data.saveArticleToStash
+import org.terst.feedreader.data.loadCachedExtraction
 import org.terst.feedreader.data.urlToArticleId
 import org.terst.feedreader.readability.ReaderExtractor
 import java.net.URLEncoder
@@ -95,9 +96,15 @@ fun LinkReaderScreen(url: String, onBack: () -> Unit, onOpenLink: (String) -> Un
     LaunchedEffect(url) {
         val existing = db.articleDao().getById(articleId)
         article = existing
-        if (existing?.extractedContent != null) {
-            extractedTitle = existing.title
-            extractedContent = existing.extractedContent
+        // A cached extraction can be the site's cookie-consent notice rather than the
+        // article (see CachedExtraction) -- drop it and extract again instead of showing
+        // a cookie disclaimer as the page.
+        val cached = loadCachedExtraction(existing?.extractedContent) {
+            db.articleDao().clearExtractedContent(articleId)
+        }
+        if (cached.content != null) {
+            extractedTitle = existing?.title
+            extractedContent = cached.content
         } else {
             val extracted = ReaderExtractor.extract(context, url)
             val content = extracted?.content
diff --git a/app/src/main/java/org/terst/feedreader/ui/ReaderScreen.kt b/app/src/main/java/org/terst/feedreader/ui/ReaderScreen.kt
index 6240b77..f08088b 100644
--- a/app/src/main/java/org/terst/feedreader/ui/ReaderScreen.kt
+++ b/app/src/main/java/org/terst/feedreader/ui/ReaderScreen.kt
@@ -68,6 +68,7 @@ import org.terst.feedreader.data.Settings
 import org.terst.feedreader.data.ensureStandaloneArticle
 import org.terst.feedreader.data.saveArticleToStash
 import org.terst.feedreader.data.setReaderModeEnabled
+import org.terst.feedreader.data.loadCachedExtraction
 import org.terst.feedreader.readability.ReaderExtractor
 import java.net.URLEncoder
 
@@ -219,15 +220,26 @@ private fun ReaderContent(
         feed = loadedFeed
 
         feedContent = loaded.feedContentHtml
-        readerContent = loaded.extractedContent
+        // A previously-cached extraction can be the site's cookie-consent notice rather
+        // than the article (reported live on a New Republic feature). That is dropped
+        // from the cache on read, which leaves nothing to show -- so re-extract, or the
+        // repair would just turn a wrong article into a blank one.
+        val cachedExtraction = loadCachedExtraction(loaded.extractedContent) {
+            db.articleDao().clearExtractedContent(articleId)
+        }
+        readerContent = cachedExtraction.content
 
         // Reader mode is NEVER heuristic/automatic (no more "content looks thin, switch
         // for you") -- it strictly follows the feed's persisted, explicitly-set opt-in.
         // Once a feed has it on, articles from it default to the extracted view and
         // auto-extract on open (that's the point of persisting it per-feed: no re-tapping
-        // per article), but nothing here ever turns it on by itself.
+        // per article), but nothing here ever turns it on by itself. Replacing a discarded
+        // consent notice does not turn it on either -- for a feed-less saved article there
+        // is no feed to opt in, and the reader falls back to the extracted content anyway.
         if (loadedFeed?.readerModeEnabled == true) {
             isReaderMode = true
+        }
+        if (loadedFeed?.readerModeEnabled == true || cachedExtraction.discardedConsentNotice) {
             if (readerContent == null) {
                 isExtracting = true
                 val result = ReaderExtractor.extractResult(context, loaded.link)
@@ -509,6 +521,10 @@ private fun ReaderContent(
                             appendLine("Session isReaderMode: $isReaderMode")
                             appendLine("isSaved: ${a?.isSaved}, stashArtifactId: ${a?.stashArtifactId}")
                             appendLine("extractedContent cached: ${readerContent != null}")
+                            // Without this, a report about bad/absent reader content could
+                            // not say whether extraction had failed, let alone how -- which
+                            // is exactly what the cookie-disclaimer report ran into.
+                            appendLine("Last extraction error: ${extractionError ?: "none"}")
                         }
                         val clipboard = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
                         clipboard.setPrimaryClip(ClipData.newPlainText("Bug report", report))