Play the YouTube embed with a referrer instead of in an iframe

Peter, on the previous build: "the inline player now plays the audio but
the video is a black screen." So Error 153 is gone and the stream is
decoding -- what is left is a display problem.

The iframe was not needed in the first place. A `Referer` header alone
satisfies the embed, which means the player can be the WebView's own
ordinary top-level document again. That drops two constructs the last
build added: a `loadDataWithBaseURL` page, whose origin is synthetic,
and a cross-origin subframe inside it. Both have a history of rendering
video oddly in WebView, and the black screen appeared with them.

Measured on an Android user agent -- no referrer: Error 153; `&origin=`
in the query: Error 153; inside an iframe: plays; with a `Referer`
header: plays. The referrer is a constant on a reserved `.invalid`
domain rather than the article's URL, so YouTube learns which app asked
and nothing about what is being read -- the same reason the nocookie
domain is used.

Confirmed in the app's own WebView: the embed loads with no error,
854x480, readyState 4, playing.

NOT confirmed: that a picture now reaches the screen. This emulator
cannot render video in a WebView at all -- it hard-crashed the AVD
twice, including on a plain <video> in an ordinary non-dialog WebView --
so the one thing that matters here can only be checked on real
hardware.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

3 files changed, 52 insertions, 70 deletions

diff --git a/app/src/main/java/org/terst/feedreader/ui/ReaderWebView.kt b/app/src/main/java/org/terst/feedreader/ui/ReaderWebView.kt
index 78020a9..1bc1315 100644
--- a/app/src/main/java/org/terst/feedreader/ui/ReaderWebView.kt
+++ b/app/src/main/java/org/terst/feedreader/ui/ReaderWebView.kt
@@ -398,17 +398,11 @@ fun YouTubePlayerDialog(videoId: String, onDismiss: () -> Unit) {
                         }
                     }
                     webChromeClient = WebChromeClient()
-                    // A host page with the embed in an iframe, not the embed itself as the
-                    // top-level document. YouTube answers a top-level embed with "Video player
-                    // configuration error -- Error 153", which is the reported "YouTube play is
-                    // misconfigured"; framed, the same URL plays. See YouTubeLinks.
-                    loadDataWithBaseURL(
-                        YOUTUBE_EMBED_BASE_URL,
-                        youTubeEmbedPageHtml(videoId),
-                        "text/html",
-                        "utf-8",
-                        null
-                    )
+                    // The embed as this WebView's own ordinary document, with a Referer
+                    // header. Without one YouTube answers "Video player configuration error --
+                    // Error 153", the reported "YouTube play is misconfigured". See
+                    // YouTubeLinks for the alternatives measured and why this one won.
+                    loadUrl(youTubeEmbedUrl(videoId), mapOf("Referer" to YOUTUBE_EMBED_REFERRER))
                 }
             },
             onRelease = { it.destroy() }
diff --git a/app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt b/app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt
index e2b3b62..00624a1 100644
--- a/app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt
+++ b/app/src/main/java/org/terst/feedreader/ui/YouTubeLinks.kt
@@ -1,26 +1,30 @@
 package org.terst.feedreader.ui
 
 /**
- * Recognising a YouTube placeholder link, and building a page its embed will actually play in.
+ * Recognising a YouTube placeholder link, and loading an embed that actually plays.
  *
  * Split out of ReaderWebView for the usual reason: this is string work with no WebView, no
  * gestures and no Compose in it, so it can be exercised without a device -- which matters here
  * because the bug it fixes is invisible until a video is tapped.
  *
- * **Why the embed needs a page at all.** The player dialog used to call
- * `loadUrl("https://www.youtube-nocookie.com/embed/<id>")`, loading the embed as the WebView's
- * own top-level document. YouTube refuses that: the player comes up showing "Video player
- * configuration error -- Error 153", which is reported as "YouTube play is misconfigured".
- * Confirmed by loading the same URL three ways in a real browser on an Android user agent:
+ * **Why the embed needs a referrer.** The player dialog called
+ * `loadUrl("https://www.youtube-nocookie.com/embed/<id>")` and YouTube answered with "Video
+ * player configuration error -- Error 153", reported as "YouTube play is misconfigured". The
+ * embed refuses to play for a page it cannot attribute to a site. Measured on an Android user
+ * agent:
  *
- * | how the embed is loaded                  | result                      |
- * |------------------------------------------|-----------------------------|
- * | top-level, as the app did                | Error 153                   |
- * | top-level with `&origin=`                | Error 153                   |
- * | in an `<iframe>` on an ordinary https page | plays                     |
+ * | how the embed is loaded                    | result    |
+ * |--------------------------------------------|-----------|
+ * | plain, no referrer                         | Error 153 |
+ * | with `&origin=` in the query               | Error 153 |
+ * | inside an `<iframe>` on an ordinary page   | plays     |
+ * | with a `Referer` header                    | plays     |
  *
- * An embed is meant to be embedded. Given a host page it behaves exactly as it does on any
- * site that includes a video, so that is what [youTubeEmbedPageHtml] provides.
+ * The last is what this uses. The iframe works too and was tried first, but it meant hosting
+ * the embed in a `loadDataWithBaseURL` document -- a synthetic origin, with the video in a
+ * cross-origin subframe of it. Both are WebView constructs with a history of rendering video
+ * oddly, and neither is needed: one request header gets the same result with the embed as the
+ * WebView's own ordinary top-level document.
  */
 
 /** Video id characters, per YouTube: 11 of these and nothing else. */
@@ -42,14 +46,14 @@ private val WATCH_QUERY = Regex("[?&]v=([A-Za-z0-9_-]{11})(?:&|$)")
 private val SHORT_PATH = Regex("^/([A-Za-z0-9_-]{11})/?$")
 
 /**
- * The origin the embed is hosted on.
+ * The site the embed is told it is playing on.
  *
- * Deliberately not a youtube.com domain: the embed checks that it is framed by a page other
- * than its own, which is the condition the top-level load failed. Nothing is ever fetched from
- * this address -- it is the base URL for `loadDataWithBaseURL`, so it only has to be a
- * well-formed https origin for the document to have one.
+ * A constant, and a deliberately uninformative one: YouTube learns which app asked and nothing
+ * else. The article's own URL would satisfy the embed just as well and would tell YouTube what
+ * is being read, which is the thing choosing the nocookie domain already avoids. `.invalid` is
+ * reserved by RFC 2606, so this can never collide with somebody's real site.
  */
-const val YOUTUBE_EMBED_BASE_URL = "https://reader.feedreader.invalid/"
+const val YOUTUBE_EMBED_REFERRER = "https://reader.feedreader.invalid/"
 
 /**
  * The video a long-press reported, as a watch URL, or null when the long-press was not on one.
@@ -83,21 +87,14 @@ fun youTubeTargetForLongPress(url: String): String? {
 fun youTubeWatchUrl(videoId: String): String = "https://www.youtube.com/watch?v=${scrub(videoId)}"
 
 /**
- * A minimal page holding nothing but the player, filling it.
+ * Where the player lives.
  *
- * `allow="autoplay"` is required for the autoplay in the src to be honoured inside an iframe --
- * without it the embed loads and then sits there, which would look like a different bug with
- * the same symptom.
+ * `youtube-nocookie.com` rather than `youtube.com`: YouTube's own privacy-enhanced embed domain,
+ * so a casual view doesn't pull in the viewer's signed-in account and history. Must be loaded
+ * with [YOUTUBE_EMBED_REFERRER] as the `Referer` header or it refuses to play.
  */
-fun youTubeEmbedPageHtml(videoId: String): String {
-    val id = scrub(videoId)
-    return """<!doctype html><html><head><meta name="viewport" content="width=device-width,initial-scale=1">""" +
-        """<style>html,body{margin:0;padding:0;height:100%;background:#000;overflow:hidden}""" +
-        """iframe{display:block;border:0;width:100%;height:100%}</style></head><body>""" +
-        """<iframe src="https://www.youtube-nocookie.com/embed/$id?autoplay=1&playsinline=1&rel=0" """ +
-        """allow="autoplay; encrypted-media; picture-in-picture; fullscreen" allowfullscreen></iframe>""" +
-        """</body></html>"""
-}
+fun youTubeEmbedUrl(videoId: String): String =
+    "https://www.youtube-nocookie.com/embed/${scrub(videoId)}?autoplay=1&playsinline=1&rel=0"
 
 /**
  * The id with anything outside YouTube's own alphabet removed.
diff --git a/app/src/test/java/org/terst/feedreader/ui/YouTubeLinksTest.kt b/app/src/test/java/org/terst/feedreader/ui/YouTubeLinksTest.kt
index 3afba16..af9884d 100644
--- a/app/src/test/java/org/terst/feedreader/ui/YouTubeLinksTest.kt
+++ b/app/src/test/java/org/terst/feedreader/ui/YouTubeLinksTest.kt
@@ -90,46 +90,37 @@ class YouTubeLinksTest {
         assertNull(youTubeTargetForLongPress("not a url at all"))
     }
 
-    // --- the embed page ------------------------------------------------------------------
+    // --- the embed ----------------------------------------------------------------------
 
     @Test
-    fun `the embed page puts the player in an iframe, which is the whole point`() {
-        val html = youTubeEmbedPageHtml("dQw4w9WgXcQ")
+    fun `the embed url is the privacy-preserving nocookie one, playing inline`() {
+        val url = youTubeEmbedUrl("dQw4w9WgXcQ")
 
-        assertTrue("loading the embed top-level is what YouTube rejects: $html", html.contains("<iframe"))
-        assertTrue(html.contains("https://www.youtube-nocookie.com/embed/dQw4w9WgXcQ"))
+        assertTrue(url.startsWith("https://www.youtube-nocookie.com/embed/dQw4w9WgXcQ"))
+        assertTrue(url.contains("autoplay=1"))
+        assertTrue(url.contains("playsinline=1"))
     }
 
     @Test
-    fun `the embed page keeps autoplay and inline playback`() {
-        val html = youTubeEmbedPageHtml("dQw4w9WgXcQ")
-
-        assertTrue(html.contains("autoplay=1"))
-        assertTrue(html.contains("playsinline=1"))
-        assertTrue("without this the iframe is not allowed to autoplay", html.contains("allow=\"autoplay"))
+    fun `the embed is sent a referrer, which is the whole reason it plays`() {
+        assertTrue(YOUTUBE_EMBED_REFERRER.startsWith("https://"))
     }
 
     @Test
-    fun `the base URL is a real https origin, since that is what the embed requires`() {
-        assertTrue(YOUTUBE_EMBED_BASE_URL.startsWith("https://"))
-    }
-
-    @Test
-    fun `the base URL is not YouTube's own origin`() {
-        assertFalse(
-            "the embed has to be a guest on somebody else's page, not its own",
-            YOUTUBE_EMBED_BASE_URL.contains("youtube")
-        )
+    fun `the referrer says nothing about what is being read`() {
+        // A constant, so YouTube learns the app and nothing else. The article's own URL would
+        // work just as well and would tell YouTube what he reads, which is the thing the
+        // nocookie domain is already chosen to avoid.
+        assertFalse(YOUTUBE_EMBED_REFERRER.contains("?"))
+        assertTrue("a reserved TLD can never become somebody's real site", YOUTUBE_EMBED_REFERRER.contains(".invalid"))
     }
 
     @Test
-    fun `a video id that could break out of the attribute is scrubbed`() {
-        val html = youTubeEmbedPageHtml("""a"><script>alert(1)</script>""")
+    fun `a video id that could break out of the url is scrubbed`() {
+        val url = youTubeEmbedUrl("""a"><script>alert(1)</script>""")
 
-        assertFalse("markup from the id reached the page: $html", html.contains("<script"))
-        assertFalse(html.contains("alert(1)"))
-        // What is left is only ever id characters, so the src attribute still closes where it should.
-        assertTrue(html.contains("embed/ascriptalert1script?"))
+        assertFalse("markup from the id reached the url: $url", url.contains("<"))
+        assertTrue(url.startsWith("https://www.youtube-nocookie.com/embed/ascriptalert1script?"))
     }
 
     @Test