|
Three times in one session I reported a build or test result that was about
something other than what I thought I was testing:
- A failed `cd` meant Gradle ran in the previous directory; the "BUILD
SUCCESSFUL" I reported was for the wrong tree.
- `adb shell monkey` + a single `pidof` sample reported a healthy app as
crashed and a crashed app as healthy, on the same emulator minutes apart.
- `adb install` refused a filename and printed so to stdout, which was
redirected to /dev/null; the old build stayed installed and got tested.
All three share a cause: concluding from console text rather than from exit
codes and structured artifacts. These wrappers make that mistake hard.
gradle-verify pins the project with -p from an absolutely-resolved argument,
treats a missing dir as fatal rather than falling back to cwd,
and reports from the exit code.
gradle-test parses JUnit XML for counts, and fails on zero tests executed,
on no XML at all, and on results whose mtime predates the run
(Gradle reporting UP-TO-DATE and skipping the task).
android-launch-check
resolves the launcher activity from the package manager rather
than guessing, starts logcat capture before launching, uses
`am start -W`, samples the pid across a settle window so a
process that starts and then dies is a failure, and prints the
fatal stack. Optionally installs an APK first and verifies the
install actually reported Success.
Two bugs of exactly this class were found and fixed in the scripts themselves
while testing them, both of which broke the crash-detection path specifically:
`... | grep -q` under pipefail SIGPIPEs its upstream and reports failure on a
successful match, and `pid=$(pidof ...)` under `set -e` aborts the script at the
moment the process is gone.
Verified against the real projects: nav/android-app 241 tests, nav/test-runner
122, feedreader 39, and both the pass and crash paths of android-launch-check on
a live emulator. gradle-test immediately caught nav reporting UP-TO-DATE with
33 stale result files, which naively read as "241 tests pass" without running
one.
12 self-tests, mostly negative cases.
|
