feedreader.git/app/src/androidTest, branch master

Render an article's maths instead of showing its TeX source

2026-10-16T21:04:38+00:00

Reported against lcamtuf's Substack ("we should render tex", blog.coredump.cx/p/yours-truly): reader mode showed raw "\(p = \text{...}\)" where equations should be. What the page actually ships is the TeX SOURCE, as text, inside a "latex-rendered" div -- Substack renders it in the browser with its own script, which never runs here. Readability keeps that text verbatim (it strips the class, not the text), so nothing was being discarded during extraction and no amount of extraction-side repair would have helped: there was simply never any rendered markup to keep. So render it during extraction, where JavaScript is already running for Readability's sake. KaTeX (vendored in assets, never fetched -- this runs against untrusted third-party pages) converts each expression to MathML before Readability sees the document, and the reader WebView renders that natively. The reader keeps JavaScript OFF, as it must, and needs no stylesheet, webfont or script of KaTeX's; the only reader-side change is containing a wide equation so it scrolls within its own box instead of dragging the whole article sideways. Two details worth knowing: - KaTeX refuses {align} outside display mode, and Substack wraps display maths in INLINE delimiters (a block-styled span does the centring), so a mode that renders nothing is retried in the other one. An expression neither mode can render is left as source -- no worse than before. - The reported article came with "extractedContent cached: true", so the raw source was already on disk and every open read it back. Recognised and dropped on read (same mechanism as the cached consent notice), which re-extracts the article the next time it is opened. APK grows by ~90KB: the vendored katex.min.js (267KB) compresses to 91KB in the APK. No new permissions, no schema change, no new JS-enabled surface. Verified on the emulator against the live article: stale raw-TeX cache discarded, article re-extracted, all four equations rendered, and a wide align block scrolls inside itself.

Fix intermittent extraction failures, and stop them being silent

2026-09-20T17:39:04+00:00

Bug report: extraction failed on one 1500days.com article while others worked. It is not that article. Extraction was racing the page load, and the report could say nothing more because the app discarded the reason. Two separate defects. 1. The race. onPageFinished fires before the page has finished loading — observed at progress=25 on this article — and the extraction script was injected there. Sometimes the DOM is far enough along; sometimes evaluateJavascript hands back a bare "null" because the document is still being replaced. Measured against the reported URL: 4 successes in 6 attempts before, 10 in 10 after. An unusable result is no longer final. A null or empty-content result is retried (6 attempts, 500ms apart, well inside the existing 25s timeout); a Readability error still fails immediately, since retrying will not change what the parser thinks of the markup. 2. The silence. ReaderScreen did `extracted?.content?.let { }`, so every failure — dead host, HTTP error, timeout, parser throw — arrived as the same null and was dropped. The reader just kept showing raw feed content. The only way a failure ever surfaced was the user noticing. extractResult() now returns a typed ExtractionFailure carrying the reason; ReaderScreen logs the diagnostic and shows the user a short explanation. Also replaced a dead error callback. The only override was the four-argument onReceivedError, which Android stopped calling at API 23; minSdk is 26, so it never fired on any supported device. A failed load therefore invoked nothing and hung for the full 25 seconds before surfacing as an indistinguishable null. Now overrides the request/error and onReceivedHttpError forms Android actually calls, both filtered to the main frame so a 404'd tracker cannot abort an article that renders fine. The instrumented test does NOT reproduce the race and says so in its own docstring: a MockWebServer page finishes loading before onPageFinished fires, and slow subresources do not help because WebView waits for them. It is kept as a reliability guard over repeated back-to-back extractions. The race was verified by hand against the live URL. 47 unit tests, 1 instrumented test, both green.

Add instrumented extraction fixture tests for WHT/Ars Technica

2026-08-24T08:10:17+00:00

BACKLOG's "Extraction test coverage for WHT / Ars Technica" item: ReaderExtractor's site-specific preprocessing had only ever been diagnosed by hand against single live articles across several rounds of fixes. Four hand-authored fixtures now pin down the exact behavior: - West Hawaii Today's paywall-div unhide (class="paywall paywall_*") - Lazy-loaded image src resolution (data-src -> src) - Ars Technica's duplicate PhotoSwipe caption removal (.pswp-caption-content vs. the real

) - Ars's split .post-content.post-content-double block merge Needs a real WebView to run Readability.js, so these are instrumented tests (app/src/androidTest/), not plain JVM unit tests -- that source set didn't exist yet (testInstrumentationRunner + androidx.test deps added to app/build.gradle.kts). Fixtures are minimal hand-authored markup targeting the exact selectors ReaderExtractor's own code uses, not scraped live pages, which rot and would fail for reasons unrelated to the extraction logic. Fixtures are served over real http:// via MockWebServer, since ReaderExtractor's WebView has allowFileAccess = false and won't load file:// URLs. Added a debug-only network security config override (app/src/debug/res/xml/network_security_config.xml) permitting cleartext to 127.0.0.1/localhost for this -- release is untouched. Verified the tests actually catch regressions, not just pass trivially: temporarily broke the paywall-unhide selector, confirmed the corresponding test failed on both connected emulators, then reverted and confirmed green again. Ran on 2 connected devices (feedreader_test, doot_test_api36): 4/4 passing, 0 failures. Instrumented tests aren't counted by the JVM-only jacoco coverage metric (expected, not a gate miss -- see CLAUDE.md). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_017GMEkHeqKz6FLkmizowBTK