summaryrefslogtreecommitdiff
path: root/app/src
diff options
context:
space:
mode:
authorPeter Stone <thepeterstone@gmail.com>2026-10-05 09:54:59 +0000
committerPeter Stone <thepeterstone@gmail.com>2026-10-05 09:54:59 +0000
commit217c982403a105e986fb7ce13b922d9f08583632 (patch)
tree38f2d3a9556b3dfa1b835b9b2c1f7efcb547e719 /app/src
parent7cc88394458e9d03e3a6a6e58cdff9c8dd4ad7a3 (diff)
Move skip detection to a server, and find that it detects ads
The server is Peter's design and it works: listend polls the phone's feed list, mirrors the newest 8 episodes per show, detects across all of them at once, and serves the offsets *and the audio they belong to*. Deployed on tungsten behind Apache at https://listen.terst.org. FingerprintWorker and MidrollWorker are deleted -- the battery drain is gone because the work is. Serving the audio is load-bearing. Dynamic ad insertion turned out to be on every episode: Grass Apes declares 22:05 and the server was served 28:18. An offset measured on one copy is minutes wrong for another, so the phone has to play the bytes the detector examined. The offsets land in the same columns the on-device detector wrote, so playback needed no changes and there is no migration. **The headline finding is a failure, and it is proved rather than suspected.** Detection reported the intro confirmed on 8 of 8 Radiolab episodes and passed its self-check. It had locked onto an advertisement. The chosen segment was searched for in an ad-free copy of the same episode fetched hours earlier: r = 0.0621. It is in the ad break, not the programme. The confirmation threshold assumes three *independently fetched* episodes. Mirroring 8 within one minute from the same ad server makes them maximally dependent -- whatever campaign is running is in all of them, identical -- and SelfCheck cannot catch it, because the segments genuinely are the same audio. Moving to a server fixed the cost problem and broke the independence the safety mechanism rests on. The published offsets are head-window matches, so they happen to skip pre-roll advertising, which is useful by accident and will move whenever the campaign rotates. It is not intro detection and is not described as such anywhere. The fix, not made: record when each episode was fetched and require the agreeing episodes to span more than one fetch occasion. Reasoning, and an argument for detecting the ads directly instead, in docs/detection-findings.md. Also recorded there: two tooling traps that each cost real time. `android-ui tap` can land on the soft keyboard, because the dump reports layout coordinates the IME does not move -- a Save button and a text field both appeared tapped and neither fired. And the server's token is alphanumeric now because `adb shell input text` ate the leading "-" of a base64url one and produced a silent 401 that looked like broken wiring. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B
Diffstat (limited to 'app/src')
-rw-r--r--app/src/main/java/org/terst/listen/data/Preferences.kt18
-rw-r--r--app/src/main/java/org/terst/listen/data/Repositories.kt36
-rw-r--r--app/src/main/java/org/terst/listen/data/SubscriptionRepository.kt52
-rw-r--r--app/src/main/java/org/terst/listen/download/EpisodeDownloader.kt60
-rw-r--r--app/src/main/java/org/terst/listen/server/OkHttpSkipTransport.kt50
-rw-r--r--app/src/main/java/org/terst/listen/server/ServerSkips.kt56
-rw-r--r--app/src/main/java/org/terst/listen/server/SkipServer.kt144
-rw-r--r--app/src/main/java/org/terst/listen/ui/MainActivity.kt27
-rw-r--r--app/src/main/java/org/terst/listen/ui/SettingsScreen.kt60
-rw-r--r--app/src/main/java/org/terst/listen/ui/SkipServerSettingsState.kt35
-rw-r--r--app/src/main/java/org/terst/listen/work/Workers.kt80
-rw-r--r--app/src/test/java/org/terst/listen/data/SubscriptionRepositoryTest.kt8
-rw-r--r--app/src/test/java/org/terst/listen/download/EpisodeDownloaderTest.kt123
-rw-r--r--app/src/test/java/org/terst/listen/server/OkHttpSkipTransportTest.kt113
-rw-r--r--app/src/test/java/org/terst/listen/server/ServerSkipsTest.kt155
-rw-r--r--app/src/test/java/org/terst/listen/server/SkipServerTest.kt194
16 files changed, 1096 insertions, 115 deletions
diff --git a/app/src/main/java/org/terst/listen/data/Preferences.kt b/app/src/main/java/org/terst/listen/data/Preferences.kt
index 1f7e14f..0c170a1 100644
--- a/app/src/main/java/org/terst/listen/data/Preferences.kt
+++ b/app/src/main/java/org/terst/listen/data/Preferences.kt
@@ -2,6 +2,7 @@ package org.terst.listen.data
import android.content.Context
import androidx.datastore.preferences.core.booleanPreferencesKey
+import androidx.datastore.preferences.core.stringPreferencesKey
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.preferencesDataStore
import kotlinx.coroutines.flow.Flow
@@ -31,7 +32,24 @@ class Preferences(private val context: Context) {
context.store.edit { it[SHOW_PLAYED] = value }
}
+ /**
+ * Where the skip server is, and the token for it. Blank means not configured, which is a
+ * working state: the app downloads from publishers and plays without detected skips,
+ * exactly as it did before the server existed.
+ */
+ val serverUrl: Flow<String> = context.store.data.map { it[SERVER_URL] ?: "" }
+ val serverToken: Flow<String> = context.store.data.map { it[SERVER_TOKEN] ?: "" }
+
+ suspend fun setServer(url: String, token: String) {
+ context.store.edit {
+ it[SERVER_URL] = url.trim()
+ it[SERVER_TOKEN] = token.trim()
+ }
+ }
+
private companion object {
val SHOW_PLAYED = booleanPreferencesKey("show_played")
+ val SERVER_URL = stringPreferencesKey("server_url")
+ val SERVER_TOKEN = stringPreferencesKey("server_token")
}
}
diff --git a/app/src/main/java/org/terst/listen/data/Repositories.kt b/app/src/main/java/org/terst/listen/data/Repositories.kt
index 1b174cf..e425ccf 100644
--- a/app/src/main/java/org/terst/listen/data/Repositories.kt
+++ b/app/src/main/java/org/terst/listen/data/Repositories.kt
@@ -3,7 +3,12 @@ package org.terst.listen.data
import android.content.Context
import coil3.ImageLoader
import coil3.network.okhttp.OkHttpNetworkFetcherFactory
+import kotlinx.coroutines.flow.first
import okhttp3.OkHttpClient
+import org.terst.listen.server.OkHttpSkipTransport
+import org.terst.listen.server.ServerSkips
+import org.terst.listen.server.SkipServer
+import org.terst.listen.server.SkipServerConfig
import org.terst.listen.download.ChaquopyAudioSource
import org.terst.listen.download.EpisodeDownloader
import org.terst.listen.feed.ChapterFetcher
@@ -96,8 +101,30 @@ class AppGraph(private val context: Context) {
subscriptions,
episodes,
FeedFetcher(client),
- onDiscovered = { ruleApplier.apply(it) },
- probe = { url -> ChaquopyAudioSource(context).probe(url) },
+ SubscriptionCollaborators(
+ onDiscovered = { ruleApplier.apply(it) },
+ probe = { url -> ChaquopyAudioSource(context).probe(url) },
+ onRefreshed = { feedUrls -> skipServer.pushSubscriptions(feedUrls) },
+ ),
+ )
+ }
+
+ /**
+ * The skip server, which is where detection happens now.
+ *
+ * It moved off the phone because it could never work here: comparing several episodes'
+ * audio at once needs more memory than an Android process gets and more time than
+ * WorkManager's ten-minute ceiling allows, and the attempt burned the battery for weeks
+ * while confirming nothing.
+ */
+ val skipServer: SkipServer by lazy {
+ SkipServer(
+ config = {
+ val url = preferences.serverUrl.first()
+ val token = preferences.serverToken.first()
+ if (url.isBlank() || token.isBlank()) null else SkipServerConfig(url, token)
+ },
+ transport = OkHttpSkipTransport(client),
)
}
@@ -107,10 +134,7 @@ class AppGraph(private val context: Context) {
episodes,
libraryDir,
ChaquopyAudioSource(context),
- // Fingerprinting needs the file on disk to decode, so it can only start once a
- // download actually lands -- one job per episode rather than a periodic sweep,
- // since the file is right there and there is nothing to gain by waiting.
- onDownloaded = { episode -> Scheduling.fingerprintNow(context, episode.id) },
+ serverAudio = ServerSkips(skipServer, subscriptions, fingerprints, playbackState),
)
}
diff --git a/app/src/main/java/org/terst/listen/data/SubscriptionRepository.kt b/app/src/main/java/org/terst/listen/data/SubscriptionRepository.kt
index 9b94c75..a66441a 100644
--- a/app/src/main/java/org/terst/listen/data/SubscriptionRepository.kt
+++ b/app/src/main/java/org/terst/listen/data/SubscriptionRepository.kt
@@ -30,26 +30,43 @@ private const val SAVED_TITLE = "Saved videos"
private const val SAVED_FACEBOOK_FEED_URL = "listen:saved:facebook"
private const val SAVED_FACEBOOK_TITLE = "Saved Facebook videos"
-class SubscriptionRepository(
- private val subscriptions: SubscriptionDao,
- private val episodes: EpisodeDao,
- private val fetcher: FeedFetcher,
+/**
+ * Everything [SubscriptionRepository] tells the rest of the app about, bundled.
+ *
+ * Grouped rather than passed one by one so the constructor stays a readable, fixed size as more
+ * of the app comes to care what a refresh found -- the same reasoning as [RuleEngine]'s and
+ * [org.terst.listen.fingerprint.MidrollUpdater]'s own tuning values.
+ */
+data class SubscriptionCollaborators(
/**
* Told about episodes a refresh has just discovered, so the auto-queue rules can act on them.
*
- * A collaborator rather than a call into the queue, so this class never learns what a queue
- * is. Deliberately not invoked by [subscribe] or [addSingleEpisode]: see their comments.
+ * Deliberately not invoked by `subscribe` or `addSingleEpisode`: see their comments.
*/
- private val onDiscovered: suspend (List<EpisodeEntity>) -> Unit = {},
+ val onDiscovered: suspend (List<EpisodeEntity>) -> Unit = {},
/**
* Asks the extractor what a video is, for the cases where its page will not say.
*
* Facebook serves no `og:` tags to a non-browser at all, so a shared video would otherwise
* be filed as "Video 10153231379946729". yt-dlp knows, and is going to be run against the
- * URL anyway. A collaborator rather than a direct call because it reaches the embedded
- * Python, which cannot run in a unit test.
+ * URL anyway.
+ */
+ val probe: suspend (String) -> ProbedVideo? = { null },
+ /**
+ * Told the subscription list on every refresh, so the skip server knows which feeds to
+ * mirror and analyse.
+ *
+ * Fired from `refreshAll`, the choke point every refresh route passes through -- including
+ * the library's own refresh button, which does not go via a worker.
*/
- private val probe: suspend (String) -> ProbedVideo? = { null },
+ val onRefreshed: suspend (List<String>) -> Unit = {},
+)
+
+class SubscriptionRepository(
+ private val subscriptions: SubscriptionDao,
+ private val episodes: EpisodeDao,
+ private val fetcher: FeedFetcher,
+ private val collaborators: SubscriptionCollaborators = SubscriptionCollaborators(),
private val now: () -> Long = System::currentTimeMillis,
) {
@@ -108,8 +125,15 @@ class SubscriptionRepository(
}
}
- suspend fun refreshAll(): Map<String, RefreshOutcome> =
- subscriptions.getAll().associate { it.id to refresh(it) }
+ suspend fun refreshAll(): Map<String, RefreshOutcome> {
+ val shows = subscriptions.getAll()
+ // The one place every refresh passes through, which is why the server is told here
+ // rather than in a worker: the refresh button in the library calls this directly and
+ // would otherwise never announce a new subscription. Failure is ignored on purpose --
+ // the skip server improves playback rather than being a precondition for it.
+ collaborators.onRefreshed(shows.map { it.feedUrl })
+ return shows.associate { it.id to refresh(it) }
+ }
/**
* Pulls a show's back catalogue by walking its RFC 5005 `prev-archive` chain.
@@ -156,7 +180,7 @@ class SubscriptionRepository(
val result = EpisodeMatcher.match(subscriptionId, stored, feed.episodes, now())
if (result.inserts.isNotEmpty()) episodes.insertAll(result.inserts)
if (result.updates.isNotEmpty()) episodes.updateAll(result.updates)
- if (report && result.inserts.isNotEmpty()) onDiscovered(result.inserts)
+ if (report && result.inserts.isNotEmpty()) collaborators.onDiscovered(result.inserts)
return RefreshOutcome.Updated(result.inserts.size, result.updates.size)
}
@@ -192,7 +216,7 @@ class SubscriptionRepository(
// Only when the page gave nothing: a title that was shared is the one that was meant,
// and this costs a network round trip.
- val probed = if (video.title.isNullOrBlank()) probe(video.watchUrl) else null
+ val probed = if (video.title.isNullOrBlank()) collaborators.probe(video.watchUrl) else null
val guid = if (facebook) "fb:video:${video.videoId}" else "yt:video:${video.videoId}"
val existing = episodes.getByGuid(container.id, guid)
diff --git a/app/src/main/java/org/terst/listen/download/EpisodeDownloader.kt b/app/src/main/java/org/terst/listen/download/EpisodeDownloader.kt
index ed8bf70..32180c4 100644
--- a/app/src/main/java/org/terst/listen/download/EpisodeDownloader.kt
+++ b/app/src/main/java/org/terst/listen/download/EpisodeDownloader.kt
@@ -9,6 +9,7 @@ import org.terst.listen.data.EpisodeDao
import org.terst.listen.data.EpisodeEntity
import org.terst.listen.data.ProbedVideo
import org.terst.listen.data.SubscriptionKind
+import org.terst.listen.server.ServerEpisode
import java.io.File
import java.io.IOException
@@ -20,18 +21,35 @@ import java.io.IOException
* play with no indication why -- so the kind decides the route, and there is no fallback
* between them.
*/
+/**
+ * The skip server's part in a download: where the audio should come from, and what to write
+ * down once it has arrived.
+ *
+ * Behind an interface because the decision belongs to the download path while the HTTP and the
+ * database writes do not.
+ */
+interface ServerAudio {
+ /** What the server has for this episode, or null to download from the publisher. */
+ suspend fun resolve(episode: EpisodeEntity): ServerEpisode?
+
+ /** Records the server's offsets, once its audio is actually the file on disk. */
+ suspend fun record(episode: EpisodeEntity, info: ServerEpisode)
+}
+
class EpisodeDownloader(
private val client: OkHttpClient,
private val episodes: EpisodeDao,
private val libraryDir: File,
private val extractor: ExtractedAudioSource? = null,
/**
- * Fired once a download actually lands on disk -- the trigger for Tier 3 fingerprint
- * matching, which needs the file present to decode. Every download route ends up here
- * regardless of whether it was a manual tap or a queued/rule-triggered batch, so this is
- * the one place that guarantees every newly downloaded episode gets checked.
+ * The skip server, preferred over the publisher when it has the episode.
+ *
+ * Not an optimisation. A show using dynamic ad insertion serves a different file on every
+ * request -- one real episode measured 22:05 on one fetch and 28:18 on another -- so an
+ * offset computed on the server's copy points at the wrong moment in any other copy.
+ * Playing the bytes the detector examined is the only thing that makes the offsets true.
*/
- private val onDownloaded: suspend (EpisodeEntity) -> Unit = {},
+ private val serverAudio: ServerAudio? = null,
private val now: () -> Long = System::currentTimeMillis,
) {
@@ -47,13 +65,15 @@ class EpisodeDownloader(
SubscriptionKind.FACEBOOK_VIDEO,
-> downloadViaExtractor(episode)
- SubscriptionKind.PODCAST -> downloadOverHttp(episode)
+ SubscriptionKind.PODCAST -> downloadPodcast(episode)
}
result.fold(
+ // Nothing is triggered by a completed download any more. It used to start Tier 3
+ // fingerprinting, which has moved to the server -- where comparing several
+ // episodes at once is possible at all.
onSuccess = {
episodes.updateDownload(episode.id, DownloadState.DOWNLOADED, it.absolutePath, now())
- onDownloaded(episode.copy(downloadState = DownloadState.DOWNLOADED, localPath = it.absolutePath))
},
// localPath is cleared: a FAILED row pointing at a path invites something downstream
// to try playing a file that is not there.
@@ -72,13 +92,35 @@ class EpisodeDownloader(
return File(path).takeIf { it.isFile && it.length() > 0 }
}
+ /**
+ * Prefers the skip server, falling back to the publisher.
+ *
+ * The fallback is deliberate and matters: skips improve playback rather than being a
+ * precondition for it, so a server that is down or mid-fetch must not turn a reachable
+ * episode into a failed download. Offsets are recorded only when the server's audio is
+ * actually what landed on disk.
+ */
+ private suspend fun downloadPodcast(episode: EpisodeEntity): Result<File> {
+ val fromServer = serverAudio?.resolve(episode)
+ val serverUrl = fromServer?.audioUrl?.takeIf { fromServer.ready }
+
+ if (serverUrl != null) {
+ val result = downloadOverHttp(episode, serverUrl)
+ if (result.isSuccess) {
+ serverAudio?.record(episode, fromServer)
+ return result
+ }
+ }
+ return downloadOverHttp(episode, episode.mediaUrl)
+ }
+
private suspend fun downloadViaExtractor(episode: EpisodeEntity): Result<File> {
val source = extractor
?: return Result.failure(IllegalStateException("No audio extractor configured"))
return runCatching { source.download(episode.mediaUrl, libraryDir) }
}
- private suspend fun downloadOverHttp(episode: EpisodeEntity): Result<File> =
+ private suspend fun downloadOverHttp(episode: EpisodeEntity, url: String): Result<File> =
withContext(Dispatchers.IO) {
val target = targetFile(episode)
// Written to a partial name and renamed on success. A truncated file recorded as
@@ -87,7 +129,7 @@ class EpisodeDownloader(
partial.parentFile?.mkdirs()
try {
- val request = Request.Builder().url(episode.mediaUrl).header("User-Agent", USER_AGENT).build()
+ val request = Request.Builder().url(url).header("User-Agent", USER_AGENT).build()
client.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
partial.delete()
diff --git a/app/src/main/java/org/terst/listen/server/OkHttpSkipTransport.kt b/app/src/main/java/org/terst/listen/server/OkHttpSkipTransport.kt
new file mode 100644
index 0000000..1541dfb
--- /dev/null
+++ b/app/src/main/java/org/terst/listen/server/OkHttpSkipTransport.kt
@@ -0,0 +1,50 @@
+package org.terst.listen.server
+
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.withContext
+import okhttp3.MediaType.Companion.toMediaType
+import okhttp3.OkHttpClient
+import okhttp3.Request
+import okhttp3.RequestBody.Companion.toRequestBody
+
+/**
+ * [SkipTransport] over the app's own OkHttp client.
+ *
+ * On the framework-adapter exclusion list: there is no decision in here, only the HTTP that
+ * [SkipServer]'s logic is deliberately kept free of.
+ *
+ * Every failure is a null or a false rather than an exception. The server is an improvement the
+ * app prefers, not something it depends on -- so unreachable, slow and broken all have to mean
+ * the same thing to the caller as not configured: carry on without it.
+ */
+class OkHttpSkipTransport(private val client: OkHttpClient) : SkipTransport {
+
+ override suspend fun get(url: String, token: String): String? = withContext(Dispatchers.IO) {
+ runCatching {
+ val request = Request.Builder()
+ .url(url)
+ .header("Authorization", "Bearer $token")
+ .build()
+ client.newCall(request).execute().use { response ->
+ if (!response.isSuccessful) return@use null
+ response.body?.string()
+ }
+ }.getOrNull()
+ }
+
+ override suspend fun post(url: String, token: String, body: String): Boolean =
+ withContext(Dispatchers.IO) {
+ runCatching {
+ val request = Request.Builder()
+ .url(url)
+ .header("Authorization", "Bearer $token")
+ .post(body.toRequestBody(JSON))
+ .build()
+ client.newCall(request).execute().use { it.isSuccessful }
+ }.getOrDefault(false)
+ }
+
+ private companion object {
+ val JSON = "application/json; charset=utf-8".toMediaType()
+ }
+}
diff --git a/app/src/main/java/org/terst/listen/server/ServerSkips.kt b/app/src/main/java/org/terst/listen/server/ServerSkips.kt
new file mode 100644
index 0000000..edff77b
--- /dev/null
+++ b/app/src/main/java/org/terst/listen/server/ServerSkips.kt
@@ -0,0 +1,56 @@
+package org.terst.listen.server
+
+import org.terst.listen.data.EpisodeEntity
+import org.terst.listen.data.PlaybackStateDao
+import org.terst.listen.data.ShowFingerprintDao
+import org.terst.listen.data.SubscriptionDao
+import org.terst.listen.download.ServerAudio
+
+/**
+ * Connects the skip server to the download path.
+ *
+ * What the server returns goes into exactly the columns the on-device detector used to write,
+ * which is why playback needed no changes at all: `effectiveIntroMs`, `effectiveOutroMs` and
+ * `PlayheadPolicy` already read them. Only the origin of the numbers has moved.
+ */
+class ServerSkips(
+ private val server: SkipServer,
+ private val subscriptions: SubscriptionDao,
+ private val fingerprints: ShowFingerprintDao,
+ private val playbackState: PlaybackStateDao,
+) : ServerAudio {
+
+ /**
+ * Asks the server about an episode, identifying it the way the server keys them.
+ *
+ * Three identifiers are sent, not one, because publishers rewrite feeds: a GUID that has
+ * moved still matches on its enclosure URL, and failing that on its title. An episode that
+ * loses its identity loses its offsets.
+ */
+ override suspend fun resolve(episode: EpisodeEntity): ServerEpisode? {
+ val feedUrl = subscriptions.getById(episode.subscriptionId)?.feedUrl ?: return null
+ return server.episode(
+ feedUrl = feedUrl,
+ guid = episode.guid,
+ enclosureUrl = episode.mediaUrl,
+ title = episode.title,
+ )
+ }
+
+ /**
+ * Writes down what the server knew, once its audio is the file on disk.
+ *
+ * Each value is written only if the server actually reported it. A missing offset is not a
+ * zero: a detected zero means "this show has no intro", which playback treats differently
+ * from having nothing detected, and collapsing the two would make a real finding
+ * indistinguishable from an absent one.
+ */
+ override suspend fun record(episode: EpisodeEntity, info: ServerEpisode) {
+ info.introMs?.let { fingerprints.setDetectedIntroMs(episode.id, it) }
+ info.outroMs?.let { fingerprints.setDetectedOutroMs(episode.id, it) }
+ // The length the file really is. The feed's figure is minutes short on any show using
+ // dynamic ad insertion, and this is the number that places the outro offset and decides
+ // when an episode counts as finished.
+ info.durationMs?.let { playbackState.updateDuration(episode.id, it) }
+ }
+}
diff --git a/app/src/main/java/org/terst/listen/server/SkipServer.kt b/app/src/main/java/org/terst/listen/server/SkipServer.kt
new file mode 100644
index 0000000..a480fb5
--- /dev/null
+++ b/app/src/main/java/org/terst/listen/server/SkipServer.kt
@@ -0,0 +1,144 @@
+package org.terst.listen.server
+
+import kotlinx.serialization.SerialName
+import kotlinx.serialization.Serializable
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.JsonPrimitive
+
+/**
+ * Where the skip server is and how to authenticate to it.
+ *
+ * Null everywhere it appears means "not configured", which is a working state: the app falls
+ * back to downloading from publishers with no detected skips, exactly as it did before the
+ * server existed.
+ */
+data class SkipServerConfig(val baseUrl: String, val token: String)
+
+/** The HTTP the client needs, behind an interface so its logic is testable off-device. */
+interface SkipTransport {
+ /** Returns the response body, or null for any failure. */
+ suspend fun get(url: String, token: String): String?
+
+ /** Returns whether the post succeeded. */
+ suspend fun post(url: String, token: String, body: String): Boolean
+}
+
+/** What the server knows about one episode. */
+data class ServerEpisode(
+ /** Whether the audio can be played from the server now. */
+ val ready: Boolean,
+ /** An absolute, token-carrying URL, usable by the player as given. */
+ val audioUrl: String?,
+ /** The file's real length, which is routinely not what the feed declared. */
+ val durationMs: Long?,
+ val introMs: Long?,
+ val outroMs: Long?,
+ /**
+ * Whether detection has run over this episode's show. Distinct from having offsets: a show
+ * can be examined and genuinely have no recurring intro, and reporting that as "nothing
+ * found yet" is the mistake the app's own readout used to make.
+ */
+ val detected: Boolean,
+)
+
+/**
+ * The phone's half of the skip server.
+ *
+ * Detection itself moved off the phone. The intro search compares several episodes' audio at
+ * once, which needs more memory than an Android process is given and more time than
+ * WorkManager's ten-minute ceiling allows -- the attempt burned the battery for weeks and
+ * confirmed nothing. What is left here is asking.
+ *
+ * The server serves the audio as well as the offsets, and that is load-bearing rather than
+ * convenient: a show using dynamic ad insertion returns a different file on every request, so
+ * an offset measured on the server's copy points at the wrong moment in a copy the phone
+ * fetched for itself. Measured on one real episode: 22:05 on one fetch, 28:18 on another.
+ * Playing the server's bytes is what makes its numbers true.
+ */
+class SkipServer(
+ private val config: suspend () -> SkipServerConfig?,
+ private val transport: SkipTransport,
+) {
+
+ /**
+ * Tells the server what the phone subscribes to.
+ *
+ * The phone owns this list: everything already built around it -- the share sheet, OPML
+ * import, the queue rules, Android Auto's browse tree -- keeps working untouched, and the
+ * server stays something the app prefers rather than depends on.
+ */
+ suspend fun pushSubscriptions(feedUrls: List<String>): Boolean {
+ val target = config() ?: return false
+ val body = buildString {
+ append("""{"feeds":[""")
+ feedUrls.forEachIndexed { index, url ->
+ if (index > 0) append(",")
+ append(JsonPrimitive(url))
+ }
+ append("]}")
+ }
+ return transport.post(base(target) + "/api/subscriptions", target.token, body)
+ }
+
+ /**
+ * Asks what to skip in one episode, and where its audio is.
+ *
+ * Null whenever there is no answer -- not configured, unreachable, or a response that is not
+ * what this expects. Every one of those means the same thing to the caller: carry on the way
+ * the app worked before the server existed.
+ */
+ suspend fun episode(
+ feedUrl: String,
+ guid: String,
+ enclosureUrl: String,
+ title: String,
+ ): ServerEpisode? {
+ val target = config() ?: return null
+ val query = buildString {
+ append("?feed=").append(encode(feedUrl))
+ append("&guid=").append(encode(guid))
+ append("&enclosure=").append(encode(enclosureUrl))
+ append("&title=").append(encode(title))
+ }
+
+ val body = transport.get(base(target) + "/api/episode" + query, target.token) ?: return null
+ val parsed = runCatching { lenient.decodeFromString<EpisodeResponse>(body) }.getOrNull() ?: return null
+
+ return ServerEpisode(
+ ready = parsed.status == "ready" && parsed.audioUrl != null,
+ audioUrl = parsed.audioUrl?.let { absolute(base(target), it) },
+ durationMs = parsed.durationMs?.takeIf { it > 0 },
+ introMs = parsed.introMs,
+ outroMs = parsed.outroMs,
+ detected = parsed.detected,
+ )
+ }
+
+ private fun base(target: SkipServerConfig) = target.baseUrl.trimEnd('/')
+
+ /**
+ * The server returns a root-relative audio path; a player handed that unresolved would try
+ * to open a local file.
+ */
+ private fun absolute(base: String, path: String): String =
+ if (path.startsWith("http://") || path.startsWith("https://")) path else base + path
+
+ @Serializable
+ private data class EpisodeResponse(
+ val status: String = "",
+ @SerialName("audioUrl") val audioUrl: String? = null,
+ @SerialName("durationMs") val durationMs: Long? = null,
+ @SerialName("introMs") val introMs: Long? = null,
+ @SerialName("outroMs") val outroMs: Long? = null,
+ val detected: Boolean = false,
+ )
+
+ private companion object {
+ /** Tolerant of fields this app does not know about, so the server can grow. */
+ val lenient = Json { ignoreUnknownKeys = true; isLenient = true }
+ }
+}
+
+/** Percent-encoding for a query value. A feed URL and a GUID both contain characters that
+ * would otherwise end the parameter early. */
+private fun encode(value: String): String = java.net.URLEncoder.encode(value, "UTF-8")
diff --git a/app/src/main/java/org/terst/listen/ui/MainActivity.kt b/app/src/main/java/org/terst/listen/ui/MainActivity.kt
index a4cc048..771e844 100644
--- a/app/src/main/java/org/terst/listen/ui/MainActivity.kt
+++ b/app/src/main/java/org/terst/listen/ui/MainActivity.kt
@@ -16,6 +16,8 @@ import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.navigation.compose.NavHost
+import androidx.navigation.NavGraphBuilder
+import androidx.navigation.NavHostController
import androidx.navigation.compose.composable
import androidx.navigation.compose.rememberNavController
@@ -108,10 +110,7 @@ private fun ListenApp(
PlayerScreen(player = player, onBack = { nav.popBackStack() })
}
episodeRoute(nav, viewModel, player)
- composable("settings") {
- SettingsScreen(opml = opmlViewModel, player = player,
- onOpenRules = { nav.navigate("rules") }, onBack = { nav.popBackStack() })
- }
+ settingsRoute(nav, opmlViewModel, player)
rulesRoutes(nav, rulesViewModel)
composable("queue") {
QueueScreen(
@@ -216,3 +215,23 @@ private fun androidx.navigation.NavGraphBuilder.rulesRoutes(
}
}
}
+
+/**
+ * The settings route, extracted so the app's own navigation graph stays readable as screens are
+ * added to it.
+ */
+private fun NavGraphBuilder.settingsRoute(
+ nav: NavHostController,
+ opml: OpmlViewModel,
+ player: PlayerViewModel,
+) {
+ composable("settings") {
+ SettingsScreen(
+ opml = opml,
+ player = player,
+ server = rememberSkipServerSettings(),
+ onOpenRules = { nav.navigate("rules") },
+ onBack = { nav.popBackStack() },
+ )
+ }
+}
diff --git a/app/src/main/java/org/terst/listen/ui/SettingsScreen.kt b/app/src/main/java/org/terst/listen/ui/SettingsScreen.kt
index 1d8da1b..9afa3e9 100644
--- a/app/src/main/java/org/terst/listen/ui/SettingsScreen.kt
+++ b/app/src/main/java/org/terst/listen/ui/SettingsScreen.kt
@@ -15,11 +15,12 @@ import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material3.Button
import androidx.compose.material3.ExperimentalMaterial3Api
+import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
-import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
+import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Snackbar
import androidx.compose.material3.SnackbarHost
@@ -30,7 +31,9 @@ import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
+import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
+import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
@@ -47,6 +50,7 @@ private val OPML_MIME_TYPES = arrayOf("text/x-opml", "text/xml", "application/xm
fun SettingsScreen(
opml: OpmlViewModel,
player: PlayerViewModel,
+ server: SkipServerSettings,
onOpenRules: () -> Unit,
onBack: () -> Unit,
) {
@@ -103,10 +107,64 @@ fun SettingsScreen(
onImport = { importLauncher.launch(OPML_MIME_TYPES) },
onExport = { exportLauncher.launch("listen-subscriptions.opml") },
)
+ HorizontalDivider()
+ SkipServerSection(server)
}
}
}
+/** The skip server's address and token, as the settings screen reads and writes them. */
+data class SkipServerSettings(
+ val url: String,
+ val token: String,
+ val onSave: (url: String, token: String) -> Unit,
+)
+
+/**
+ * Where the skip detection comes from.
+ *
+ * Detection used to run on the phone and could not work there: comparing several episodes'
+ * audio at once needs more memory than an Android process gets and more time than WorkManager
+ * allows, so it burned the battery for weeks and confirmed nothing. The server does that work
+ * and serves the audio it examined -- which is the part that matters, because a show using
+ * dynamic ad insertion returns a different file on every request and an offset measured on one
+ * copy is wrong for another.
+ *
+ * Left blank the app behaves as it did before: downloads from publishers, no detected skips.
+ */
+@Composable
+private fun SkipServerSection(settings: SkipServerSettings) {
+ var url by remember(settings.url) { mutableStateOf(settings.url) }
+ var token by remember(settings.token) { mutableStateOf(settings.token) }
+
+ Section("Skip server") {
+ OutlinedTextField(
+ value = url,
+ onValueChange = { url = it },
+ label = { Text("Address") },
+ placeholder = { Text("https://listen.terst.org") },
+ singleLine = true,
+ modifier = Modifier.fillMaxWidth(),
+ )
+ OutlinedTextField(
+ value = token,
+ onValueChange = { token = it },
+ label = { Text("Token") },
+ singleLine = true,
+ modifier = Modifier.fillMaxWidth(),
+ )
+ Button(onClick = { settings.onSave(url, token) }) { Text("Save") }
+ Text(
+ if (settings.url.isBlank()) {
+ "Not set. Episodes download from publishers and play without detected skips."
+ } else {
+ "Episodes download from the server, which is also what analysed them."
+ },
+ style = MaterialTheme.typography.bodySmall,
+ )
+ }
+}
+
@Composable
private fun SpeedSection(current: Float, onSelect: (Float) -> Unit) {
Section("Playback speed") {
diff --git a/app/src/main/java/org/terst/listen/ui/SkipServerSettingsState.kt b/app/src/main/java/org/terst/listen/ui/SkipServerSettingsState.kt
new file mode 100644
index 0000000..fc0e49d
--- /dev/null
+++ b/app/src/main/java/org/terst/listen/ui/SkipServerSettingsState.kt
@@ -0,0 +1,35 @@
+package org.terst.listen.ui
+
+import androidx.compose.runtime.Composable
+import androidx.compose.runtime.collectAsState
+import androidx.compose.runtime.getValue
+import androidx.compose.runtime.remember
+import androidx.compose.runtime.rememberCoroutineScope
+import androidx.compose.ui.platform.LocalContext
+import kotlinx.coroutines.launch
+import org.terst.listen.data.Graph
+
+/**
+ * Reads and writes the skip server's address and token.
+ *
+ * Trivial glue, kept out of the composable that renders them so the screen stays a screen. In
+ * DataStore rather than Room, like the other settings that describe how the app is being used
+ * rather than what is in the library.
+ */
+@Composable
+internal fun rememberSkipServerSettings(): SkipServerSettings {
+ val context = LocalContext.current
+ val preferences = remember(context) { Graph.get(context).preferences }
+ val scope = rememberCoroutineScope()
+
+ val url by preferences.serverUrl.collectAsState(initial = "")
+ val token by preferences.serverToken.collectAsState(initial = "")
+
+ return SkipServerSettings(
+ url = url,
+ token = token,
+ onSave = { newUrl, newToken ->
+ scope.launch { preferences.setServer(newUrl, newToken) }
+ },
+ )
+}
diff --git a/app/src/main/java/org/terst/listen/work/Workers.kt b/app/src/main/java/org/terst/listen/work/Workers.kt
index 9c48db0..9809e9d 100644
--- a/app/src/main/java/org/terst/listen/work/Workers.kt
+++ b/app/src/main/java/org/terst/listen/work/Workers.kt
@@ -12,9 +12,6 @@ import androidx.work.WorkManager
import androidx.work.WorkerParameters
import org.terst.listen.data.Graph
import org.terst.listen.data.RefreshOutcome
-import org.terst.listen.fingerprint.FingerprintUpdater
-import org.terst.listen.fingerprint.MediaCodecClipSource
-import org.terst.listen.fingerprint.MidrollUpdater
import java.util.concurrent.TimeUnit
/**
@@ -28,6 +25,9 @@ class RefreshWorker(context: Context, params: WorkerParameters) : CoroutineWorke
override suspend fun doWork(): Result {
val graph = Graph.get(applicationContext)
+ // The subscription list reaches the skip server from SubscriptionRepository.refreshAll,
+ // which every refresh route goes through -- including the library's refresh button,
+ // which does not come via this worker.
val outcomes = graph.repository.refreshAll()
val failed = outcomes.values.count { it is RefreshOutcome.Failed }
// After the refresh, so a drip release can pick up anything that just arrived.
@@ -77,62 +77,15 @@ class RetentionWorker(context: Context, params: WorkerParameters) : CoroutineWor
}
}
-/**
- * Checks one newly downloaded episode against its show's Tier 3 fingerprint state.
- *
- * The decode is [org.terst.listen.fingerprint.MediaCodecClipSource] (no JVM equivalent); the
- * decision is [org.terst.listen.fingerprint.FingerprintUpdater], which is tested.
- */
-class FingerprintWorker(context: Context, params: WorkerParameters) : CoroutineWorker(context, params) {
-
- override suspend fun doWork(): Result {
- val episodeId = inputData.getString(KEY_EPISODE_ID) ?: return Result.failure()
- val graph = Graph.get(applicationContext)
- val episode = graph.episodes.getById(episodeId) ?: return Result.success()
-
- FingerprintUpdater(graph.fingerprints, graph.episodes, MediaCodecClipSource()).process(episode)
-
- return Result.success()
- }
-
- companion object {
- const val KEY_EPISODE_ID = "episodeId"
- }
-}
-
-/**
- * Runs [MidrollUpdater] for every show, nightly rather than per-download like [FingerprintWorker]:
- * a full-body scan is expensive enough (see the desktop prototype findings) that it is worth
- * batching, and unlike a download it never touches the network, so there is nothing per-episode
- * to react to promptly.
- */
-class MidrollWorker(context: Context, params: WorkerParameters) : CoroutineWorker(context, params) {
-
- override suspend fun doWork(): Result {
- val graph = Graph.get(applicationContext)
- val updater = MidrollUpdater(
- graph.subscriptions,
- graph.episodes,
- graph.midrolls,
- MediaCodecClipSource(),
- graph.midrollClips,
- )
- for (show in graph.subscriptions.getAll()) updater.processShow(show.id)
- return Result.success()
- }
-}
-
object Scheduling {
private const val REFRESH_WORK = "listen-refresh"
private const val DOWNLOAD_WORK = "listen-download"
private const val DOWNLOAD_NOW_WORK = "listen-download-now"
private const val RETENTION_WORK = "listen-retention"
- private const val MIDROLL_WORK = "listen-midroll"
private const val REFRESH_HOURS = 6L
private const val DOWNLOAD_HOURS = 2L
private const val RETENTION_HOURS = 12L
- private const val MIDROLL_HOURS = 24L
/**
* Fetches what is in the queue now, rather than at the next periodic sweep.
@@ -150,20 +103,6 @@ object Scheduling {
}
/**
- * Checks one freshly downloaded episode against its show's fingerprint state.
- *
- * Not a unique work name: unlike a burst of queueing, a run of downloads is one episode
- * each, and there is no "latest one wins" -- every episode needs its own check.
- */
- fun fingerprintNow(context: Context, episodeId: String) {
- WorkManager.getInstance(context).enqueue(
- OneTimeWorkRequestBuilder<FingerprintWorker>()
- .setInputData(androidx.work.Data.Builder().putString(FingerprintWorker.KEY_EPISODE_ID, episodeId).build())
- .build(),
- )
- }
-
- /**
* The periodic jobs are KEEP rather than UPDATE, so the schedule survives an app restart
* instead of resetting its interval every launch -- which on a frequently-opened app means
* the periodic work never actually fires.
@@ -208,18 +147,5 @@ object Scheduling {
.build(),
)
- manager.enqueueUniquePeriodicWork(
- MIDROLL_WORK,
- ExistingPeriodicWorkPolicy.KEEP,
- PeriodicWorkRequestBuilder<MidrollWorker>(MIDROLL_HOURS, TimeUnit.HOURS)
- .setConstraints(
- Constraints.Builder()
- // Charging, not wifi: everything it touches is already on disk, so
- // unlike DownloadWorker there is no data cost to gate on network for.
- .setRequiresCharging(true)
- .build()
- )
- .build(),
- )
}
}
diff --git a/app/src/test/java/org/terst/listen/data/SubscriptionRepositoryTest.kt b/app/src/test/java/org/terst/listen/data/SubscriptionRepositoryTest.kt
index ccede65..2474f8a 100644
--- a/app/src/test/java/org/terst/listen/data/SubscriptionRepositoryTest.kt
+++ b/app/src/test/java/org/terst/listen/data/SubscriptionRepositoryTest.kt
@@ -349,7 +349,7 @@ class SubscriptionRepositoryTest {
subs,
eps,
FeedFetcher(OkHttpClient()),
- probe = { ProbedVideo(title = "Sheep dog at work", durationMs = 92_000) },
+ SubscriptionCollaborators(probe = { ProbedVideo(title = "Sheep dog at work", durationMs = 92_000) }),
) { clock }
probing.addSingleEpisode(video("111", null, kind = SubscriptionKind.FACEBOOK_VIDEO))
@@ -366,7 +366,7 @@ class SubscriptionRepositoryTest {
subs,
eps,
FeedFetcher(OkHttpClient()),
- probe = { ProbedVideo(title = "Something else", durationMs = 1) },
+ SubscriptionCollaborators(probe = { ProbedVideo(title = "Something else", durationMs = 1) }),
) { clock }
probing.addSingleEpisode(video("aaa", "From the page"))
@@ -381,7 +381,7 @@ class SubscriptionRepositoryTest {
subs,
eps,
FeedFetcher(OkHttpClient()),
- probe = { null },
+ SubscriptionCollaborators(probe = { null }),
) { clock }
probing.addSingleEpisode(video("111", null, kind = SubscriptionKind.FACEBOOK_VIDEO))
@@ -403,7 +403,7 @@ class SubscriptionRepositoryTest {
subs,
eps,
FeedFetcher(OkHttpClient()),
- onDiscovered = { discovered += it },
+ SubscriptionCollaborators(onDiscovered = { discovered += it }),
) { clock }
@Test
diff --git a/app/src/test/java/org/terst/listen/download/EpisodeDownloaderTest.kt b/app/src/test/java/org/terst/listen/download/EpisodeDownloaderTest.kt
index d5e971b..b35983a 100644
--- a/app/src/test/java/org/terst/listen/download/EpisodeDownloaderTest.kt
+++ b/app/src/test/java/org/terst/listen/download/EpisodeDownloaderTest.kt
@@ -17,6 +17,7 @@ import org.terst.listen.data.DownloadState
import org.terst.listen.data.EpisodeEntity
import org.terst.listen.data.FakeEpisodeDao
import org.terst.listen.data.SubscriptionKind
+import org.terst.listen.server.ServerEpisode
import java.io.File
class EpisodeDownloaderTest {
@@ -161,4 +162,126 @@ class EpisodeDownloaderTest {
downloader.download(ep, SubscriptionKind.YOUTUBE_CHANNEL)
assertEquals("no plain HTTP request should have been made", 0, server.requestCount)
}
+
+ // ---- the skip server as the preferred source ----
+ //
+ // The server has to serve the audio, not just the offsets. A show using dynamic ad
+ // insertion returns a different file on every request -- one real episode measured 22:05 on
+ // one fetch and 28:18 on another -- so an offset computed on the server's copy is simply
+ // wrong for a copy the phone fetched itself. Downloading from the server is what makes its
+ // numbers apply to the bytes being played.
+
+ private class FakeServerAudio(private val info: ServerEpisode?) : ServerAudio {
+ var recorded: Pair<String, ServerEpisode>? = null
+ override suspend fun resolve(episode: EpisodeEntity) = info
+ override suspend fun record(episode: EpisodeEntity, info: ServerEpisode) {
+ recorded = episode.id to info
+ }
+ }
+
+ private fun withServer(audio: ServerAudio?) =
+ EpisodeDownloader(OkHttpClient(), dao, library, null, audio) { 1_000L }
+
+ @Test
+ fun `downloads from the skip server when it has the episode`() = runTest {
+ server.enqueue(MockResponse().setBody("server-bytes"))
+ val audio = FakeServerAudio(
+ ServerEpisode(
+ ready = true,
+ audioUrl = server.url("/audio/abc?token=t").toString(),
+ durationMs = 1_698_717,
+ introMs = 142_000,
+ outroMs = 11_000,
+ detected = true,
+ )
+ )
+ val ep = episode(url = "https://publisher.example/original.mp3").also { dao.rows += it }
+
+ val result = withServer(audio).download(ep, SubscriptionKind.PODCAST)
+
+ assertTrue(result.isSuccess)
+ assertEquals("server-bytes", result.getOrThrow().readText())
+ assertEquals("/audio/abc?token=t", server.takeRequest().path)
+ }
+
+ @Test
+ fun `records what the server knew once the file is on disk`() = runTest {
+ server.enqueue(MockResponse().setBody("server-bytes"))
+ val info = ServerEpisode(
+ ready = true,
+ audioUrl = server.url("/audio/abc").toString(),
+ durationMs = 1_698_717,
+ introMs = 142_000,
+ outroMs = 11_000,
+ detected = true,
+ )
+ val audio = FakeServerAudio(info)
+ val ep = episode(url = "https://publisher.example/original.mp3").also { dao.rows += it }
+
+ withServer(audio).download(ep, SubscriptionKind.PODCAST)
+
+ assertEquals(ep.id to info, audio.recorded)
+ }
+
+ @Test
+ fun `downloads from the publisher when the server has nothing yet`() = runTest {
+ server.enqueue(MockResponse().setBody("publisher-bytes"))
+ val ep = episode(url = server.url("/original.mp3").toString()).also { dao.rows += it }
+
+ val result = withServer(FakeServerAudio(null)).download(ep, SubscriptionKind.PODCAST)
+
+ assertTrue(result.isSuccess)
+ assertEquals("/original.mp3", server.takeRequest().path)
+ }
+
+ @Test
+ fun `records nothing when the server had nothing`() = runTest {
+ server.enqueue(MockResponse().setBody("publisher-bytes"))
+ val audio = FakeServerAudio(null)
+ val ep = episode(url = server.url("/original.mp3").toString()).also { dao.rows += it }
+
+ withServer(audio).download(ep, SubscriptionKind.PODCAST)
+
+ assertNull(audio.recorded)
+ }
+
+ /**
+ * The server going down must not cost Peter the episode. Skips are an improvement on
+ * playback, not a precondition for it -- and a dependency that turns a reachable publisher
+ * into a failed download would make the app worse than before the server existed.
+ */
+ @Test
+ fun `falls back to the publisher when the server's own audio fails`() = runTest {
+ server.enqueue(MockResponse().setResponseCode(502))
+ server.enqueue(MockResponse().setBody("publisher-bytes"))
+ val audio = FakeServerAudio(
+ ServerEpisode(ready = true, audioUrl = server.url("/audio/broken").toString(),
+ durationMs = null, introMs = 1, outroMs = 2, detected = true)
+ )
+ val ep = episode(url = server.url("/original.mp3").toString()).also { dao.rows += it }
+
+ val result = withServer(audio).download(ep, SubscriptionKind.PODCAST)
+
+ assertTrue("expected the publisher fallback to succeed", result.isSuccess)
+ assertEquals("publisher-bytes", result.getOrThrow().readText())
+ assertEquals("/audio/broken", server.takeRequest().path)
+ assertEquals("/original.mp3", server.takeRequest().path)
+ }
+
+ /** Offsets describe the server's copy, so they must not be recorded against a file that
+ * came from the publisher instead. */
+ @Test
+ fun `records nothing when the audio came from the publisher after all`() = runTest {
+ server.enqueue(MockResponse().setResponseCode(502))
+ server.enqueue(MockResponse().setBody("publisher-bytes"))
+ val audio = FakeServerAudio(
+ ServerEpisode(ready = true, audioUrl = server.url("/audio/broken").toString(),
+ durationMs = null, introMs = 1, outroMs = 2, detected = true)
+ )
+ val ep = episode(url = server.url("/original.mp3").toString()).also { dao.rows += it }
+
+ withServer(audio).download(ep, SubscriptionKind.PODCAST)
+
+ assertNull("recorded offsets for audio it did not download", audio.recorded)
+ }
}
diff --git a/app/src/test/java/org/terst/listen/server/OkHttpSkipTransportTest.kt b/app/src/test/java/org/terst/listen/server/OkHttpSkipTransportTest.kt
new file mode 100644
index 0000000..335b68d
--- /dev/null
+++ b/app/src/test/java/org/terst/listen/server/OkHttpSkipTransportTest.kt
@@ -0,0 +1,113 @@
+package org.terst.listen.server
+
+import kotlinx.coroutines.test.runTest
+import okhttp3.OkHttpClient
+import okhttp3.mockwebserver.MockResponse
+import okhttp3.mockwebserver.MockWebServer
+import org.junit.After
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertFalse
+import org.junit.Assert.assertNull
+import org.junit.Assert.assertTrue
+import org.junit.Before
+import org.junit.Test
+
+/**
+ * The transport's whole contract is that nothing it does can break the app.
+ *
+ * The skip server is an improvement on playback, not a precondition for it: unreachable, slow,
+ * misconfigured and returning nonsense all have to mean the same thing to the caller as "not
+ * configured" -- carry on downloading from publishers. An exception escaping from here would
+ * turn a reachable episode into a failed download, which is worse than never having had a
+ * server at all.
+ */
+class OkHttpSkipTransportTest {
+
+ private lateinit var server: MockWebServer
+ private lateinit var transport: OkHttpSkipTransport
+
+ @Before
+ fun setUp() {
+ server = MockWebServer()
+ server.start()
+ transport = OkHttpSkipTransport(OkHttpClient())
+ }
+
+ @After
+ fun tearDown() = server.shutdown()
+
+ @Test
+ fun `returns the body of a successful get`() = runTest {
+ server.enqueue(MockResponse().setBody("""{"status":"ready"}"""))
+
+ val got = transport.get(server.url("/api/episode").toString(), "tok")
+
+ assertEquals("""{"status":"ready"}""", got)
+ }
+
+ @Test
+ fun `sends the token as a bearer header`() = runTest {
+ server.enqueue(MockResponse().setBody("{}"))
+
+ transport.get(server.url("/api/episode").toString(), "hunter2")
+
+ assertEquals("Bearer hunter2", server.takeRequest().getHeader("Authorization"))
+ }
+
+ @Test
+ fun `an unauthorised response is nothing rather than an error`() = runTest {
+ server.enqueue(MockResponse().setResponseCode(401).setBody("""{"error":"unauthorized"}"""))
+
+ assertNull(transport.get(server.url("/api/episode").toString(), "wrong"))
+ }
+
+ @Test
+ fun `a server error is nothing rather than an error`() = runTest {
+ server.enqueue(MockResponse().setResponseCode(502))
+
+ assertNull(transport.get(server.url("/api/episode").toString(), "tok"))
+ }
+
+ /** A misconfigured address must not throw out of the download path. */
+ @Test
+ fun `an unreachable host is nothing rather than an error`() = runTest {
+ assertNull(transport.get("http://127.0.0.1:1/api/episode", "tok"))
+ }
+
+ @Test
+ fun `a malformed url is nothing rather than an error`() = runTest {
+ assertNull(transport.get("not a url at all", "tok"))
+ }
+
+ @Test
+ fun `posts the body as json`() = runTest {
+ server.enqueue(MockResponse().setResponseCode(200))
+
+ val ok = transport.post(
+ server.url("/api/subscriptions").toString(),
+ "tok",
+ """{"feeds":["https://a"]}""",
+ )
+
+ assertTrue(ok)
+ val request = server.takeRequest()
+ assertEquals("POST", request.method)
+ assertEquals("""{"feeds":["https://a"]}""", request.body.readUtf8())
+ assertTrue(
+ "content type was ${request.getHeader("Content-Type")}",
+ request.getHeader("Content-Type")?.startsWith("application/json") == true,
+ )
+ }
+
+ @Test
+ fun `a rejected post reports false`() = runTest {
+ server.enqueue(MockResponse().setResponseCode(400))
+
+ assertFalse(transport.post(server.url("/api/subscriptions").toString(), "tok", "{}"))
+ }
+
+ @Test
+ fun `a post to an unreachable host reports false`() = runTest {
+ assertFalse(transport.post("http://127.0.0.1:1/api/subscriptions", "tok", "{}"))
+ }
+}
diff --git a/app/src/test/java/org/terst/listen/server/ServerSkipsTest.kt b/app/src/test/java/org/terst/listen/server/ServerSkipsTest.kt
new file mode 100644
index 0000000..57c03f3
--- /dev/null
+++ b/app/src/test/java/org/terst/listen/server/ServerSkipsTest.kt
@@ -0,0 +1,155 @@
+package org.terst.listen.server
+
+import kotlinx.coroutines.test.runTest
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertNull
+import org.junit.Assert.assertTrue
+import org.junit.Test
+import org.terst.listen.data.EpisodeEntity
+import org.terst.listen.data.FakeEpisodeDao
+import org.terst.listen.data.FakePlaybackStateDao
+import org.terst.listen.data.FakeShowFingerprintDao
+import org.terst.listen.data.FakeSubscriptionDao
+import org.terst.listen.data.SubscriptionEntity
+
+/**
+ * What the phone does with the server's answer.
+ *
+ * The offsets land in the same columns the on-device detector used to write, which is why
+ * playback needed no changes at all: `effectiveIntroMs`, `MidrollSkip` and `PlayheadPolicy`
+ * already read exactly these. Only where the numbers come from has changed.
+ */
+class ServerSkipsTest {
+
+ private class StubTransport(private val body: String?) : SkipTransport {
+ val gets = mutableListOf<String>()
+ override suspend fun get(url: String, token: String): String? {
+ gets += url
+ return body
+ }
+
+ override suspend fun post(url: String, token: String, body: String) = true
+ }
+
+ private val show = SubscriptionEntity(
+ id = "s1",
+ feedUrl = "https://feeds.simplecast.com/EmVW7VGp",
+ title = "Radiolab",
+ addedAtMs = 0,
+ )
+
+ private val episode = EpisodeEntity(
+ id = "e1",
+ subscriptionId = "s1",
+ guid = "guid-1",
+ title = "Grass Apes",
+ mediaUrl = "https://cdn.example/grass.mp3",
+ mediaType = "audio/mpeg",
+ discoveredAtMs = 0,
+ )
+
+ private fun subject(
+ transport: SkipTransport,
+ subscriptions: FakeSubscriptionDao = FakeSubscriptionDao().apply { rows += show },
+ episodes: FakeEpisodeDao = FakeEpisodeDao().apply { rows += episode },
+ fingerprints: FakeShowFingerprintDao = FakeShowFingerprintDao(),
+ ): ServerSkips {
+ // Both fakes write through to the same episode rows, the way the real foreign table
+ // does, so a test sees the value where production would put it.
+ fingerprints.episodeRows = episodes.rows
+ return ServerSkips(
+ SkipServer({ SkipServerConfig("https://listen.terst.org", "tok") }, transport),
+ subscriptions,
+ fingerprints,
+ FakePlaybackStateDao(episodes.rows),
+ )
+ }
+
+ /** The server keys episodes by feed, so the show's feed URL has to be looked up and sent. */
+ @Test
+ fun `asks the server using the show's feed url`() = runTest {
+ val transport = StubTransport("""{"status":"pending"}""")
+
+ subject(transport).resolve(episode)
+
+ val url = transport.gets.single()
+ assertTrue("feed url not sent in $url", url.contains("EmVW7VGp"))
+ assertTrue("guid not sent in $url", url.contains("guid-1"))
+ }
+
+ /** An episode whose show has been unsubscribed has no feed to ask about. */
+ @Test
+ fun `does not ask when the show is gone`() = runTest {
+ val transport = StubTransport("""{"status":"pending"}""")
+
+ val got = subject(transport, subscriptions = FakeSubscriptionDao()).resolve(episode)
+
+ assertNull(got)
+ assertTrue("asked about a show that is not subscribed", transport.gets.isEmpty())
+ }
+
+ @Test
+ fun `records the detected offsets against the episode`() = runTest {
+ val episodes = FakeEpisodeDao().apply { rows += episode }
+
+ subject(StubTransport(null), episodes = episodes).record(
+ episode,
+ ServerEpisode(ready = true, audioUrl = "x", durationMs = 1_698_717,
+ introMs = 142_000, outroMs = 11_000, detected = true),
+ )
+
+ val stored = episodes.rows.single { it.id == "e1" }
+ assertEquals(142_000L, stored.detectedIntroMs)
+ assertEquals(11_000L, stored.detectedOutroMs)
+ }
+
+ /**
+ * The measured length too: the feed's claim is minutes short on any show using dynamic ad
+ * insertion, and this length is the one the file actually has.
+ */
+ @Test
+ fun `records the measured duration`() = runTest {
+ val episodes = FakeEpisodeDao().apply { rows += episode }
+
+ subject(StubTransport(null), episodes = episodes).record(
+ episode,
+ ServerEpisode(ready = true, audioUrl = "x", durationMs = 1_698_717,
+ introMs = null, outroMs = null, detected = true),
+ )
+
+ assertEquals(1_698_717L, episodes.rows.single { it.id == "e1" }.actualDurationMs)
+ }
+
+ /**
+ * An absent offset must not be written as zero. Zero is a real detected value meaning "this
+ * show has no intro", and playback distinguishes the two.
+ */
+ @Test
+ fun `leaves an offset the server did not report alone`() = runTest {
+ val episodes = FakeEpisodeDao().apply { rows += episode }
+
+ subject(StubTransport(null), episodes = episodes).record(
+ episode,
+ ServerEpisode(ready = true, audioUrl = "x", durationMs = null,
+ introMs = 142_000, outroMs = null, detected = true),
+ )
+
+ val stored = episodes.rows.single { it.id == "e1" }
+ assertEquals(142_000L, stored.detectedIntroMs)
+ assertNull("wrote an outro the server never reported", stored.detectedOutroMs)
+ }
+
+ /** A detected zero is a finding, not an absence, and has to be stored as one. */
+ @Test
+ fun `records a detected zero offset`() = runTest {
+ val episodes = FakeEpisodeDao().apply { rows += episode }
+
+ subject(StubTransport(null), episodes = episodes).record(
+ episode,
+ ServerEpisode(ready = true, audioUrl = "x", durationMs = null,
+ introMs = 0, outroMs = null, detected = true),
+ )
+
+ assertEquals(0L, episodes.rows.single { it.id == "e1" }.detectedIntroMs)
+ }
+}
diff --git a/app/src/test/java/org/terst/listen/server/SkipServerTest.kt b/app/src/test/java/org/terst/listen/server/SkipServerTest.kt
new file mode 100644
index 0000000..bdf9d6a
--- /dev/null
+++ b/app/src/test/java/org/terst/listen/server/SkipServerTest.kt
@@ -0,0 +1,194 @@
+package org.terst.listen.server
+
+import kotlinx.coroutines.test.runTest
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertNull
+import org.junit.Assert.assertTrue
+import org.junit.Test
+
+/**
+ * The phone's half of the skip server.
+ *
+ * Detection does not happen here any more, and for a reason worth keeping written down: the
+ * intro search needs several episodes' audio compared at once, which needs more memory than an
+ * Android process gets and more time than WorkManager allows. The phone's job is now to say
+ * what it subscribes to, ask what to skip, and play the audio the answer belongs to.
+ *
+ * That last part is not optional. A show using dynamic ad insertion serves a different file on
+ * every request, so an offset measured on the server's copy is wrong for a copy the phone
+ * fetched itself -- Grass Apes came back at 22:05 on one fetch and 28:18 on another. Playing
+ * the server's bytes is what makes the numbers apply.
+ */
+class SkipServerTest {
+
+ private class FakeTransport : SkipTransport {
+ var response: String? = null
+ var failPost = false
+ val gets = mutableListOf<String>()
+ val posts = mutableListOf<Pair<String, String>>()
+ var seenToken: String? = null
+
+ override suspend fun get(url: String, token: String): String? {
+ gets += url
+ seenToken = token
+ return response
+ }
+
+ override suspend fun post(url: String, token: String, body: String): Boolean {
+ posts += url to body
+ seenToken = token
+ return !failPost
+ }
+ }
+
+ private fun server(
+ transport: SkipTransport,
+ baseUrl: String? = "https://listen.terst.org",
+ token: String? = "tok",
+ ) = SkipServer({ if (baseUrl == null || token == null) null else SkipServerConfig(baseUrl, token) }, transport)
+
+ private val ready = """
+ {"status":"ready","audioUrl":"/audio/abc123?token=tok","durationMs":1698717,
+ "introMs":142000,"outroMs":11000,"detected":true}
+ """.trimIndent()
+
+ @Test
+ fun `reads what the server says about an episode`() = runTest {
+ val transport = FakeTransport().apply { response = ready }
+
+ val got = server(transport).episode("https://f", "g1", "https://cdn/1.mp3", "One")
+
+ assertTrue("expected a ready episode, got $got", got?.ready == true)
+ assertEquals("https://listen.terst.org/audio/abc123?token=tok", got?.audioUrl)
+ assertEquals(1698717L, got?.durationMs)
+ assertEquals(142000L, got?.introMs)
+ assertEquals(11000L, got?.outroMs)
+ assertTrue(got?.detected == true)
+ }
+
+ /**
+ * The server returns a root-relative audio path. Handing that to a player unresolved would
+ * have it try to open a local file.
+ */
+ @Test
+ fun `resolves a relative audio url against the server`() = runTest {
+ val transport = FakeTransport().apply {
+ response = """{"status":"ready","audioUrl":"/audio/abc?token=tok"}"""
+ }
+
+ val got = server(transport, baseUrl = "https://listen.terst.org/").episode("https://f", "g1", "", "")
+
+ assertEquals("https://listen.terst.org/audio/abc?token=tok", got?.audioUrl)
+ }
+
+ @Test
+ fun `an absolute audio url is left alone`() = runTest {
+ val transport = FakeTransport().apply {
+ response = """{"status":"ready","audioUrl":"https://cdn.example/abc.mp3"}"""
+ }
+
+ val got = server(transport).episode("https://f", "g1", "", "")
+
+ assertEquals("https://cdn.example/abc.mp3", got?.audioUrl)
+ }
+
+ /** Pending is a real answer: the server is fetching it, so the phone should not treat it
+ * as a failure, but it has nothing to play yet either. */
+ @Test
+ fun `a pending episode is reported as not ready`() = runTest {
+ val transport = FakeTransport().apply { response = """{"status":"pending","detected":false}""" }
+
+ val got = server(transport).episode("https://f", "g1", "", "")
+
+ assertEquals(false, got?.ready)
+ assertNull(got?.audioUrl)
+ }
+
+ /** Missing offsets must come back as absent, not as zero: zero is a real detected value
+ * meaning "this show has no intro", and playback treats the two differently. */
+ @Test
+ fun `absent offsets are null rather than zero`() = runTest {
+ val transport = FakeTransport().apply { response = """{"status":"ready","audioUrl":"/audio/a"}""" }
+
+ val got = server(transport).episode("https://f", "g1", "", "")
+
+ assertNull(got?.introMs)
+ assertNull(got?.outroMs)
+ }
+
+ /** A feed URL and a GUID both contain characters that would otherwise break the query. */
+ @Test
+ fun `encodes the feed url and guid into the query`() = runTest {
+ val transport = FakeTransport()
+
+ server(transport).episode(
+ feedUrl = "https://feeds.simplecast.com/EmVW7VGp?x=1&y=2",
+ guid = "tag:example,2026:1 odd",
+ enclosureUrl = "https://cdn/a b.mp3",
+ title = "Grass Apes & Co",
+ )
+
+ val url = transport.gets.single()
+ assertTrue("feed not encoded in $url", url.contains("feed=https%3A%2F%2Ffeeds.simplecast.com%2FEmVW7VGp%3Fx%3D1%26y%3D2"))
+ assertTrue("guid not encoded in $url", url.contains("guid=tag%3Aexample%2C2026%3A1+odd"))
+ assertTrue("ampersand in the title not encoded in $url", url.contains("%26+Co"))
+ }
+
+ @Test
+ fun `sends the token`() = runTest {
+ val transport = FakeTransport().apply { response = ready }
+ server(transport, token = "hunter2").episode("https://f", "g1", "", "")
+ assertEquals("hunter2", transport.seenToken)
+ }
+
+ /** With nothing configured the server is simply not used, and nothing throws. */
+ @Test
+ fun `an unconfigured server is a no-op rather than an error`() = runTest {
+ val transport = FakeTransport()
+
+ assertNull(server(transport, baseUrl = null).episode("https://f", "g1", "", ""))
+ assertEquals(false, server(transport, token = null).pushSubscriptions(listOf("https://f")))
+ assertTrue("talked to a server that is not configured", transport.gets.isEmpty() && transport.posts.isEmpty())
+ }
+
+ @Test
+ fun `rubbish from the server is reported as nothing rather than crashing`() = runTest {
+ val transport = FakeTransport().apply { response = "<html>502 Bad Gateway</html>" }
+
+ assertNull(server(transport).episode("https://f", "g1", "", ""))
+ }
+
+ @Test
+ fun `no response at all is reported as nothing`() = runTest {
+ assertNull(server(FakeTransport()).episode("https://f", "g1", "", ""))
+ }
+
+ @Test
+ fun `pushes the subscription list as json`() = runTest {
+ val transport = FakeTransport()
+
+ val ok = server(transport).pushSubscriptions(listOf("https://a", "https://b"))
+
+ assertTrue(ok)
+ val (url, body) = transport.posts.single()
+ assertEquals("https://listen.terst.org/api/subscriptions", url)
+ assertEquals("""{"feeds":["https://a","https://b"]}""", body)
+ }
+
+ @Test
+ fun `a failed push is reported rather than thrown`() = runTest {
+ val transport = FakeTransport().apply { failPost = true }
+
+ assertEquals(false, server(transport).pushSubscriptions(listOf("https://a")))
+ }
+
+ /** A trailing slash on the configured URL must not produce a double slash in the path. */
+ @Test
+ fun `tolerates a trailing slash on the base url`() = runTest {
+ val transport = FakeTransport()
+
+ server(transport, baseUrl = "https://listen.terst.org/").pushSubscriptions(listOf("https://a"))
+
+ assertEquals("https://listen.terst.org/api/subscriptions", transport.posts.single().first)
+ }
+}