summaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorPeter Stone <thepeterstone@gmail.com>2026-09-18 21:04:06 +0000
committerPeter Stone <thepeterstone@gmail.com>2026-09-18 21:04:06 +0000
commitb51ba30f1c77bcadfbf96b2eb7b4917c20faa7b0 (patch)
tree5d4d3f5c258de177d12882ffa17803e8df323ca8 /README.md
Add devtools: build/test wrappers that can't be misread
Three times in one session I reported a build or test result that was about something other than what I thought I was testing: - A failed `cd` meant Gradle ran in the previous directory; the "BUILD SUCCESSFUL" I reported was for the wrong tree. - `adb shell monkey` + a single `pidof` sample reported a healthy app as crashed and a crashed app as healthy, on the same emulator minutes apart. - `adb install` refused a filename and printed so to stdout, which was redirected to /dev/null; the old build stayed installed and got tested. All three share a cause: concluding from console text rather than from exit codes and structured artifacts. These wrappers make that mistake hard. gradle-verify pins the project with -p from an absolutely-resolved argument, treats a missing dir as fatal rather than falling back to cwd, and reports from the exit code. gradle-test parses JUnit XML for counts, and fails on zero tests executed, on no XML at all, and on results whose mtime predates the run (Gradle reporting UP-TO-DATE and skipping the task). android-launch-check resolves the launcher activity from the package manager rather than guessing, starts logcat capture before launching, uses `am start -W`, samples the pid across a settle window so a process that starts and then dies is a failure, and prints the fatal stack. Optionally installs an APK first and verifies the install actually reported Success. Two bugs of exactly this class were found and fixed in the scripts themselves while testing them, both of which broke the crash-detection path specifically: `... | grep -q` under pipefail SIGPIPEs its upstream and reports failure on a successful match, and `pid=$(pidof ...)` under `set -e` aborts the script at the moment the process is gone. Verified against the real projects: nav/android-app 241 tests, nav/test-runner 122, feedreader 39, and both the pass and crash paths of android-launch-check on a live emulator. gradle-test immediately caught nav reporting UP-TO-DATE with 33 stale result files, which naively read as "241 tests pass" without running one. 12 self-tests, mostly negative cases.
Diffstat (limited to 'README.md')
-rw-r--r--README.md88
1 files changed, 88 insertions, 0 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..e3b96c6
--- /dev/null
+++ b/README.md
@@ -0,0 +1,88 @@
+# devtools
+
+Small, shared wrappers around Gradle and adb whose only job is to make a build or
+test result **impossible to misread**.
+
+These exist because of a specific, repeated failure: concluding that something
+passed or failed by reading console text, when the text was about a different
+project, a stale result, or a command that never ran. Every script here reports
+from exit codes and structured artifacts instead, and fails loudly rather than
+defaulting to something plausible.
+
+Invoke by absolute path — `/workspace/devtools/bin/<script>`. Nothing needs to be
+on `PATH`, which is itself one less piece of ambient state to get wrong.
+
+---
+
+## The rule these encode
+
+> **Never conclude from grepped stdout. Conclude from exit codes and structured
+> artifacts.**
+
+Corollaries, each learned the hard way:
+
+| Trap | What it looks like | Guard |
+| --- | --- | --- |
+| cwd drift | `cd x && cp y .` fails; the next line runs Gradle in the old directory and prints `BUILD SUCCESSFUL` about the wrong project | `gradle-verify` takes the project dir as an argument, resolves it absolutely, and passes `-p`. A missing dir is fatal, never a fallback. |
+| Stale test results | Gradle reports `UP-TO-DATE`, runs nothing, and the XML on disk describes older code | `gradle-test` compares result mtimes against a marker written at run start and refuses stale numbers |
+| Zero tests | A test source set that won't compile leaves a green build with nothing executed | `gradle-test` fails when the parsed test count is 0 |
+| `cmd \| tail; echo $?` | Reports `tail`'s status, not the command's | Redirect to a file, then check `$?`. Or use `PIPESTATUS[0]`. |
+| `... \| grep -q` under `pipefail` | `grep -q` exits on first match and SIGPIPEs upstream; the pipeline reports failure despite matching | Do membership tests in pure shell (`case`), not pipelines |
+| `set -e` + a command that legitimately fails | `pid=$(pidof x)` aborts the whole script when the process is gone — i.e. exactly when you are trying to detect a crash | `|| true` on any substitution whose non-zero status is meaningful data |
+| Discarded command output | `adb install ... >/dev/null` hides "filename doesn't end .apk"; the old build stays installed and is silently what gets tested | `android-launch-check --apk` checks the install output for `Success` |
+| `monkey` as a launch check | Reports a healthy app dead and a dead app healthy; lands on whatever activity is foreground | `android-launch-check` uses `am start -W` against a resolved activity and samples the pid repeatedly |
+
+---
+
+## Scripts
+
+### `gradle-verify <project-dir> <task...> [-- <gradle args>]`
+
+Runs Gradle against an explicitly named project. Exits with Gradle's exit code.
+Prints the absolute project directory actually used, so the output is
+self-describing.
+
+```
+/workspace/devtools/bin/gradle-verify /workspace/nav/android-app compileDebugKotlin
+```
+
+### `gradle-test <project-dir> [task] [--allow-stale] [-- <gradle args>]`
+
+Runs the test task, then reports counts parsed from the JUnit XML. Auto-detects
+`testDebugUnitTest` for Android projects and `test` otherwise.
+
+Fails on: non-zero Gradle exit, any failure or error in the XML, **zero tests
+executed**, **no XML at all**, or **results that predate this run**.
+
+```
+/workspace/devtools/bin/gradle-test /workspace/nav/android-app -- --rerun-tasks
+```
+
+`--rerun-tasks` is usually what you want when the answer matters — otherwise
+Gradle may skip the task and the script will (correctly) refuse the stale
+results.
+
+### `android-launch-check <package> [--apk <path>] [--serial <s>] [--settle <secs>]`
+
+Answers "does this build actually start and stay up", and shows the exception if
+not. Optionally installs an APK first, verifying the install genuinely succeeded.
+
+Exit codes: `0` launched and stayed up · `1` crashed or died · `2` operator error
+(no device, package not installed, unusable filename).
+
+```
+/workspace/devtools/bin/android-launch-check org.terst.feedreader \
+ --apk /path/to/app-debug.apk --settle 15
+```
+
+---
+
+## Tests
+
+```
+/workspace/devtools/test/run-tests.sh
+```
+
+12 cases, mostly negative — each drives a script into one failure mode and
+asserts it is caught. A wrapper that only ever reports PASS is worse than no
+wrapper, so the negative cases are the point.