Un-hide CSS-gated content before running Readability

Some "soft" paywalls (West Hawaii Today's included) deliver the full
article in the initial HTML response to every visitor, then hide it with
CSS (inline or via stylesheet) pending a client-side subscription check --
the content itself isn't withheld by the server, only its visibility.
Readability has a built-in heuristic that skips anything hidden this way,
treating it as boilerplate, which is exactly what was discarding real,
already-delivered article text and leaving only the visible teaser/
subscribe prompt.

Before parsing, walk the live document (getComputedStyle needs an attached
node) and force display/visibility back on for anything hidden via
display:none or visibility:hidden, then clone and hand that off to
Readability as before.

Verified live against a real West Hawaii Today article: extractedContent
went from 588 characters (the RSS teaser) to 3458 (the full body).

1 files changed, 19 insertions, 0 deletions

diff --git a/app/src/main/java/org/terst/feedreader/readability/ReaderExtractor.kt b/app/src/main/java/org/terst/feedreader/readability/ReaderExtractor.kt
index 286588a..28c762c 100644
--- a/app/src/main/java/org/terst/feedreader/readability/ReaderExtractor.kt
+++ b/app/src/main/java/org/terst/feedreader/readability/ReaderExtractor.kt
@@ -69,6 +69,25 @@ object ReaderExtractor {
                     val script = """
                         (function() {
                             try {
+                                // Some "soft" paywalls deliver the full article in the initial HTML
+                                // response to every visitor, then hide it with CSS (inline or via a
+                                // stylesheet rule) pending a client-side subscription check -- the
+                                // content itself isn't withheld by the server, only its visibility is.
+                                // Readability has a built-in heuristic that skips anything hidden this
+                                // way (treating it as boilerplate to discard), which is exactly what
+                                // was throwing away real, already-delivered article text and leaving
+                                // only the visible teaser/subscribe prompt. Un-hide before extracting --
+                                // this must run on the live document (getComputedStyle needs a node
+                                // attached to the render tree), so it happens before the clone below.
+                                var all = document.querySelectorAll('*');
+                                for (var i = 0; i < all.length; i++) {
+                                    var el = all[i];
+                                    var cs = window.getComputedStyle(el);
+                                    if (cs.display === 'none' || cs.visibility === 'hidden') {
+                                        el.style.setProperty('display', 'block', 'important');
+                                        el.style.setProperty('visibility', 'visible', 'important');
+                                    }
+                                }
                                 $readabilityJs
                                 var article = new Readability(document.cloneNode(true)).parse();
                                 return JSON.stringify(article);