diff options
| author | Peter Stone <thepeterstone@gmail.com> | 2026-09-18 21:04:06 +0000 |
|---|---|---|
| committer | Peter Stone <thepeterstone@gmail.com> | 2026-09-18 21:04:06 +0000 |
| commit | b51ba30f1c77bcadfbf96b2eb7b4917c20faa7b0 (patch) | |
| tree | 5d4d3f5c258de177d12882ffa17803e8df323ca8 | |
Add devtools: build/test wrappers that can't be misread
Three times in one session I reported a build or test result that was about
something other than what I thought I was testing:
- A failed `cd` meant Gradle ran in the previous directory; the "BUILD
SUCCESSFUL" I reported was for the wrong tree.
- `adb shell monkey` + a single `pidof` sample reported a healthy app as
crashed and a crashed app as healthy, on the same emulator minutes apart.
- `adb install` refused a filename and printed so to stdout, which was
redirected to /dev/null; the old build stayed installed and got tested.
All three share a cause: concluding from console text rather than from exit
codes and structured artifacts. These wrappers make that mistake hard.
gradle-verify pins the project with -p from an absolutely-resolved argument,
treats a missing dir as fatal rather than falling back to cwd,
and reports from the exit code.
gradle-test parses JUnit XML for counts, and fails on zero tests executed,
on no XML at all, and on results whose mtime predates the run
(Gradle reporting UP-TO-DATE and skipping the task).
android-launch-check
resolves the launcher activity from the package manager rather
than guessing, starts logcat capture before launching, uses
`am start -W`, samples the pid across a settle window so a
process that starts and then dies is a failure, and prints the
fatal stack. Optionally installs an APK first and verifies the
install actually reported Success.
Two bugs of exactly this class were found and fixed in the scripts themselves
while testing them, both of which broke the crash-detection path specifically:
`... | grep -q` under pipefail SIGPIPEs its upstream and reports failure on a
successful match, and `pid=$(pidof ...)` under `set -e` aborts the script at the
moment the process is gone.
Verified against the real projects: nav/android-app 241 tests, nav/test-runner
122, feedreader 39, and both the pass and crash paths of android-launch-check on
a live emulator. gradle-test immediately caught nav reporting UP-TO-DATE with
33 stale result files, which naively read as "241 tests pass" without running
one.
12 self-tests, mostly negative cases.
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | README.md | 88 | ||||
| -rwxr-xr-x | bin/android-launch-check | 159 | ||||
| -rwxr-xr-x | bin/gradle-test | 140 | ||||
| -rwxr-xr-x | bin/gradle-verify | 63 | ||||
| -rwxr-xr-x | test/run-tests.sh | 161 |
6 files changed, 612 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..f47cb20 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +*.out diff --git a/README.md b/README.md new file mode 100644 index 0000000..e3b96c6 --- /dev/null +++ b/README.md @@ -0,0 +1,88 @@ +# devtools + +Small, shared wrappers around Gradle and adb whose only job is to make a build or +test result **impossible to misread**. + +These exist because of a specific, repeated failure: concluding that something +passed or failed by reading console text, when the text was about a different +project, a stale result, or a command that never ran. Every script here reports +from exit codes and structured artifacts instead, and fails loudly rather than +defaulting to something plausible. + +Invoke by absolute path — `/workspace/devtools/bin/<script>`. Nothing needs to be +on `PATH`, which is itself one less piece of ambient state to get wrong. + +--- + +## The rule these encode + +> **Never conclude from grepped stdout. Conclude from exit codes and structured +> artifacts.** + +Corollaries, each learned the hard way: + +| Trap | What it looks like | Guard | +| --- | --- | --- | +| cwd drift | `cd x && cp y .` fails; the next line runs Gradle in the old directory and prints `BUILD SUCCESSFUL` about the wrong project | `gradle-verify` takes the project dir as an argument, resolves it absolutely, and passes `-p`. A missing dir is fatal, never a fallback. | +| Stale test results | Gradle reports `UP-TO-DATE`, runs nothing, and the XML on disk describes older code | `gradle-test` compares result mtimes against a marker written at run start and refuses stale numbers | +| Zero tests | A test source set that won't compile leaves a green build with nothing executed | `gradle-test` fails when the parsed test count is 0 | +| `cmd \| tail; echo $?` | Reports `tail`'s status, not the command's | Redirect to a file, then check `$?`. Or use `PIPESTATUS[0]`. | +| `... \| grep -q` under `pipefail` | `grep -q` exits on first match and SIGPIPEs upstream; the pipeline reports failure despite matching | Do membership tests in pure shell (`case`), not pipelines | +| `set -e` + a command that legitimately fails | `pid=$(pidof x)` aborts the whole script when the process is gone — i.e. exactly when you are trying to detect a crash | `|| true` on any substitution whose non-zero status is meaningful data | +| Discarded command output | `adb install ... >/dev/null` hides "filename doesn't end .apk"; the old build stays installed and is silently what gets tested | `android-launch-check --apk` checks the install output for `Success` | +| `monkey` as a launch check | Reports a healthy app dead and a dead app healthy; lands on whatever activity is foreground | `android-launch-check` uses `am start -W` against a resolved activity and samples the pid repeatedly | + +--- + +## Scripts + +### `gradle-verify <project-dir> <task...> [-- <gradle args>]` + +Runs Gradle against an explicitly named project. Exits with Gradle's exit code. +Prints the absolute project directory actually used, so the output is +self-describing. + +``` +/workspace/devtools/bin/gradle-verify /workspace/nav/android-app compileDebugKotlin +``` + +### `gradle-test <project-dir> [task] [--allow-stale] [-- <gradle args>]` + +Runs the test task, then reports counts parsed from the JUnit XML. Auto-detects +`testDebugUnitTest` for Android projects and `test` otherwise. + +Fails on: non-zero Gradle exit, any failure or error in the XML, **zero tests +executed**, **no XML at all**, or **results that predate this run**. + +``` +/workspace/devtools/bin/gradle-test /workspace/nav/android-app -- --rerun-tasks +``` + +`--rerun-tasks` is usually what you want when the answer matters — otherwise +Gradle may skip the task and the script will (correctly) refuse the stale +results. + +### `android-launch-check <package> [--apk <path>] [--serial <s>] [--settle <secs>]` + +Answers "does this build actually start and stay up", and shows the exception if +not. Optionally installs an APK first, verifying the install genuinely succeeded. + +Exit codes: `0` launched and stayed up · `1` crashed or died · `2` operator error +(no device, package not installed, unusable filename). + +``` +/workspace/devtools/bin/android-launch-check org.terst.feedreader \ + --apk /path/to/app-debug.apk --settle 15 +``` + +--- + +## Tests + +``` +/workspace/devtools/test/run-tests.sh +``` + +12 cases, mostly negative — each drives a script into one failure mode and +asserts it is caught. A wrapper that only ever reports PASS is worse than no +wrapper, so the negative cases are the point. diff --git a/bin/android-launch-check b/bin/android-launch-check new file mode 100755 index 0000000..00dfcfd --- /dev/null +++ b/bin/android-launch-check @@ -0,0 +1,159 @@ +#!/usr/bin/env bash +# +# Determine whether an Android app actually launches and stays up, and surface +# the exception if it does not. +# +# Why this exists: `adb shell monkey -p <pkg> -c android.intent.category.LAUNCHER 1` +# followed by polling `pidof` is not a reliable check. In practice it reported a +# healthy app as dead and a dead app as healthy, in the same session, on the same +# emulator — monkey's launch can land on whatever activity happens to be in the +# foreground, and a single `pidof` sample races app startup and restart. +# +# This script instead: +# * resolves the launcher activity from the package manager rather than guessing +# * starts a continuous logcat capture BEFORE launching, so the crash window is +# never missed (the `-b crash` buffer was empty even for real FATAL crashes) +# * launches with `am start -W` and checks its reported status +# * samples the pid repeatedly over a settle period, so a process that starts +# and then dies is reported as a failure, not a success +# * greps the captured log for FATAL EXCEPTION and prints the stack +# +# Usage: +# android-launch-check <package> [--serial <adb-serial>] [--settle <seconds>] +# +# Exit: 0 = launched and stayed up with no fatal exception, 1 = did not. + +set -euo pipefail + +die() { printf '%s: %s\n' "$(basename "$0")" "$*" >&2; exit 2; } + +[ $# -ge 1 ] || die "usage: $(basename "$0") <package> [--serial <s>] [--settle <secs>]" + +pkg=$1; shift +serial="" +settle=20 +apk="" +while [ $# -gt 0 ]; do + case "$1" in + --serial) serial=${2:-}; shift 2 ;; + --settle) settle=${2:-20}; shift 2 ;; + --apk) apk=${2:-}; shift 2 ;; + *) die "unknown argument: $1" ;; + esac +done + +adb_bin=$(command -v adb || echo /opt/android-sdk/platform-tools/adb) +[ -x "$adb_bin" ] || die "adb not found" +ADB=("$adb_bin") +[ -n "$serial" ] && ADB+=(-s "$serial") + +devices=$("${ADB[@]}" devices | awk 'NR>1 && $2=="device" {print $1}') +[ -n "$devices" ] || die "no adb device in 'device' state" +[ "$(printf '%s\n' "$devices" | wc -l)" -eq 1 ] || [ -n "$serial" ] \ + || die "multiple devices attached; pass --serial. Found: $(echo $devices)" + +printf '=== android-launch-check ===\n' +printf 'package : %s\n' "$pkg" +printf 'device : %s\n' "$(echo $devices | tr '\n' ' ')" + +if [ -n "$apk" ]; then + [ -f "$apk" ] || die "apk not found: $apk" + # adb rejects any filename not ending .apk/.apex, and prints that refusal to + # stdout while still exiting 0 in some versions — so check the text as well as + # the status. Discarding this output is how a "reinstall" can silently not + # happen and leave the previous build under test. + case "$apk" in + *.apk|*.apex) : ;; + *) die "adb will refuse this filename (must end .apk/.apex): $apk" ;; + esac + printf 'apk : %s\n' "$apk" + install_out=$("${ADB[@]}" install -r "$apk" 2>&1 | tr -d '\r' || true) + if ! printf '%s' "$install_out" | grep -q '^Success$'; then + printf 'result : FAIL — install did not succeed\n' + printf '%s\n' "$install_out" | sed 's/^/ /' + exit 1 + fi + printf 'install : Success\n' +fi + +# Membership test done in pure shell, not `... | grep -q`. Under `set -o pipefail` +# a `grep -q` that exits on its first match can SIGPIPE the upstream command, and +# the pipeline then reports failure even though the match succeeded — which made +# this check report an installed package as missing. +pkg_list=$("${ADB[@]}" shell pm list packages 2>/dev/null | tr -d '\r' || true) +case $'\n'"$pkg_list"$'\n' in + *$'\n'"package:$pkg"$'\n'*) : ;; + *) die "package not installed: $pkg" ;; +esac + +# Resolve rather than assume. Guessing ".MainActivity" when the manifest says +# ".ui.MainActivity" produces "Activity class does not exist", which is an +# operator error that can easily be misread as an app crash. +activity=$("${ADB[@]}" shell cmd package resolve-activity --brief "$pkg" 2>/dev/null \ + | tr -d '\r' | tail -1) +case "$activity" in + "$pkg"/*) : ;; + *) die "could not resolve launcher activity for $pkg (got: '$activity')" ;; +esac +printf 'activity: %s\n' "$activity" + +logfile=$(mktemp); trap 'rm -f "$logfile"' EXIT + +"${ADB[@]}" shell am force-stop "$pkg" >/dev/null 2>&1 || true +sleep 2 +"${ADB[@]}" logcat -c >/dev/null 2>&1 || true + +# Capture starts first and runs across the whole window. +"${ADB[@]}" logcat -v threadtime > "$logfile" 2>&1 & +logcat_pid=$! +trap 'kill "$logcat_pid" 2>/dev/null || true; rm -f "$logfile"' EXIT +sleep 2 + +start_out=$("${ADB[@]}" shell am start -W -n "$activity" 2>&1 | tr -d '\r') +start_status=$(printf '%s\n' "$start_out" | awk -F': *' '/^Status:/ {print $2}') +printf 'am start: %s\n' "${start_status:-<none>}" + +# Sample repeatedly. One sample cannot distinguish "not started yet" from +# "started and already died". +alive_samples=0; total_samples=0; last_pid="" +interval=5 +elapsed=0 +while [ "$elapsed" -lt "$settle" ]; do + sleep "$interval" + elapsed=$((elapsed + interval)) + # `|| true` is load-bearing: pidof exits non-zero when the process is gone, + # which under `set -e` would abort this script at exactly the moment it is + # supposed to be detecting a crash. + pid=$("${ADB[@]}" shell pidof "$pkg" 2>/dev/null | tr -d '\r' | awk '{print $1}' || true) + total_samples=$((total_samples + 1)) + if [ -n "$pid" ]; then alive_samples=$((alive_samples + 1)); last_pid=$pid; fi + printf ' t=%-3ss pid=%s\n' "$elapsed" "${pid:-<none>}" +done + +sleep 1 +kill "$logcat_pid" 2>/dev/null || true +wait "$logcat_pid" 2>/dev/null || true + +fatal_count=$(grep -c "FATAL EXCEPTION" "$logfile" 2>/dev/null || true) +fatal_count=${fatal_count:-0} + +printf 'samples : %d/%d alive\n' "$alive_samples" "$total_samples" +printf 'fatal : %s\n' "$fatal_count" + +if [ "$fatal_count" -gt 0 ]; then + printf '\n--- first fatal exception ---\n' + grep -A 25 -m 1 "FATAL EXCEPTION" "$logfile" | sed 's/^/ /' + printf '\nresult : FAIL — app crashed\n' + exit 1 +fi + +if [ "$alive_samples" -ne "$total_samples" ]; then + printf '\nresult : FAIL — process was not alive for the whole settle window\n' + printf 'hint : started but died, or never started. No FATAL in logcat, so\n' + printf ' check for a native crash or an early System.exit.\n' + exit 1 +fi + +printf 'pid : %s\n' "$last_pid" +printf 'result : PASS — launched and stayed up %ss with no fatal exception\n' "$settle" +exit 0 diff --git a/bin/gradle-test b/bin/gradle-test new file mode 100755 index 0000000..5b059d7 --- /dev/null +++ b/bin/gradle-test @@ -0,0 +1,140 @@ +#!/usr/bin/env bash +# +# Run a Gradle test task and report counts parsed from the JUnit XML it wrote, +# not from console text. +# +# Catches three failure modes that all look like success from the console: +# +# 1. Zero tests ran. A unit-test source set that fails to compile, or a filter +# that matches nothing, can leave the build green with nothing executed. +# "BUILD SUCCESSFUL" and "0 tests" are the same line of output. +# 2. Stale results. If Gradle decides the test task is UP-TO-DATE it does not +# re-run anything, and the XML on disk describes an older version of the +# code. Reading those numbers as current is how a passing suite gets +# reported for code that was never tested. +# 3. Results from the wrong project, via cwd drift — handled by gradle-verify. +# +# Usage: +# gradle-test <project-dir> [test-task] [--allow-stale] [-- <gradle args>] +# +# Default task: testDebugUnitTest for Android projects, test otherwise. + +set -euo pipefail + +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +die() { printf '%s: %s\n' "$(basename "$0")" "$*" >&2; exit 2; } + +[ $# -ge 1 ] || die "usage: $(basename "$0") <project-dir> [test-task] [--allow-stale] [-- <gradle args>]" + +raw_dir=$1; shift +project_dir=$(cd "$raw_dir" 2>/dev/null && pwd) || die "no such project dir: $raw_dir" + +task="" +allow_stale=0 +extra=() +seen_sep=0 +for arg in "$@"; do + if [ "$arg" = "--" ]; then seen_sep=1; continue; fi + if [ $seen_sep -eq 1 ]; then extra+=("$arg"); continue; fi + case "$arg" in + --allow-stale) allow_stale=1 ;; + -*) extra+=("$arg") ;; + *) task=$arg ;; + esac +done + +if [ -z "$task" ]; then + if grep -rqs "com.android.application\|com.android.library" \ + "$project_dir"/build.gradle "$project_dir"/build.gradle.kts \ + "$project_dir"/*/build.gradle "$project_dir"/*/build.gradle.kts 2>/dev/null; then + task=testDebugUnitTest + else + task=test + fi +fi + +# Marker file: anything written at or after this is from the run we just did. +# Using a file rather than a clock reading avoids any filesystem/shell timestamp +# granularity mismatch. +marker=$(mktemp) +trap 'rm -f "$marker"' EXIT +sleep 1 # ensure a strictly-earlier mtime than anything the build writes + +set +e +"$here/gradle-verify" "$project_dir" "$task" ${extra[@]+-- "${extra[@]}"} +gradle_status=$? +set -e + +printf '\n=== gradle-test results ===\n' + +mapfile -t all_xml < <(find "$project_dir" -path '*/test-results/*' -name 'TEST-*.xml' 2>/dev/null | sort) +if [ ${#all_xml[@]} -eq 0 ]; then + printf 'result : FAIL — no JUnit XML found anywhere under %s\n' "$project_dir" + printf 'hint : the test task may not have run, or writes results elsewhere\n' + exit 1 +fi + +fresh=(); stale=() +for f in "${all_xml[@]}"; do + if [ "$f" -nt "$marker" ]; then fresh+=("$f"); else stale+=("$f"); fi +done + +printf 'task : %s\n' "$task" +printf 'xml : %d file(s) total, %d fresh, %d stale\n' \ + "${#all_xml[@]}" "${#fresh[@]}" "${#stale[@]}" + +if [ ${#fresh[@]} -eq 0 ]; then + if [ $allow_stale -eq 1 ]; then + printf 'note : no fresh results; counting stale ones because --allow-stale\n' + fresh=("${all_xml[@]}") + else + printf 'result : FAIL — every result file predates this run\n' + printf 'hint : Gradle treated %s as UP-TO-DATE, so these numbers describe\n' "$task" + printf ' older code. Re-run with: -- --rerun-tasks\n' + exit 1 + fi +fi + +counts=$(python3 - "${fresh[@]}" <<'PY' +import sys, xml.etree.ElementTree as ET +tests = failures = errors = skipped = 0 +bad = [] +for p in sys.argv[1:]: + try: + r = ET.parse(p).getroot() + except Exception as e: + bad.append(f"{p}: unparseable ({e})") + continue + tests += int(r.get('tests', 0)) + failures += int(r.get('failures', 0)) + errors += int(r.get('errors', 0)) + skipped += int(r.get('skipped', 0)) +print(f"{tests} {failures} {errors} {skipped}") +for b in bad: + print("UNPARSEABLE " + b, file=sys.stderr) +PY +) +read -r tests failures errors skipped <<<"$counts" + +printf 'tests : %s\n' "$tests" +printf 'failures: %s\n' "$failures" +printf 'errors : %s\n' "$errors" +printf 'skipped : %s\n' "$skipped" + +rc=0 +[ "$gradle_status" -ne 0 ] && rc=1 +[ "$failures" -gt 0 ] && rc=1 +[ "$errors" -gt 0 ] && rc=1 +if [ "$tests" -eq 0 ]; then + printf 'result : FAIL — zero tests executed\n' + printf 'hint : a green build with no tests is not a passing suite\n' + exit 1 +fi + +if [ $rc -eq 0 ]; then + printf 'result : PASS (%s tests, gradle exit %d)\n' "$tests" "$gradle_status" +else + printf 'result : FAIL (gradle exit %d, %s failures, %s errors)\n' \ + "$gradle_status" "$failures" "$errors" +fi +exit $rc diff --git a/bin/gradle-verify b/bin/gradle-verify new file mode 100755 index 0000000..73fb69d --- /dev/null +++ b/bin/gradle-verify @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# +# Run Gradle tasks against an explicitly-named project and report the result from +# the exit code, never from the console text. +# +# Why this exists: reading "BUILD SUCCESSFUL" out of piped Gradle output is not a +# reliable signal. A shell chain like `cd dir && cp f . ` followed on the next +# line by `./gradlew test` will happily run Gradle in whatever directory the +# shell was already in if the cd failed, and the success line it prints is then +# about a different project entirely. That failure mode is silent and reads +# exactly like success. +# +# Usage: +# gradle-verify <project-dir> <gradle-task...> [-- <extra gradle args>] +# +# Exits with Gradle's own exit code. Prints a summary block that names the +# absolute project directory actually used. + +set -euo pipefail + +die() { printf '%s: %s\n' "$(basename "$0")" "$*" >&2; exit 2; } + +[ $# -ge 2 ] || die "usage: $(basename "$0") <project-dir> <gradle-task...> [-- <gradle args>]" + +raw_dir=$1; shift + +# Resolve to an absolute path up front and fail loudly if it is not what we +# expect. Every later step uses this resolved value, so there is nothing for an +# ambient cwd to influence. +project_dir=$(cd "$raw_dir" 2>/dev/null && pwd) \ + || die "project dir does not exist or is not readable: $raw_dir" +[ -x "$project_dir/gradlew" ] \ + || die "no executable gradlew in: $project_dir" + +tasks=() +extra=() +seen_sep=0 +for arg in "$@"; do + if [ "$arg" = "--" ]; then seen_sep=1; continue; fi + if [ $seen_sep -eq 1 ]; then extra+=("$arg"); else tasks+=("$arg"); fi +done +[ ${#tasks[@]} -ge 1 ] || die "no gradle tasks given" + +printf '=== gradle-verify ===\n' +printf 'project : %s\n' "$project_dir" +printf 'tasks : %s\n' "${tasks[*]}" +[ ${#extra[@]} -gt 0 ] && printf 'args : %s\n' "${extra[*]}" + +# -p pins the project directory regardless of cwd. Note gradlew is invoked by +# absolute path too, so we are not depending on PATH or on cwd for the wrapper +# either. +set +e +"$project_dir/gradlew" -p "$project_dir" "${tasks[@]}" ${extra[@]+"${extra[@]}"} +status=$? +set -e + +printf 'exit : %d\n' "$status" +if [ $status -eq 0 ]; then + printf 'result : PASS\n' +else + printf 'result : FAIL\n' +fi +exit $status diff --git a/test/run-tests.sh b/test/run-tests.sh new file mode 100755 index 0000000..3a8117d --- /dev/null +++ b/test/run-tests.sh @@ -0,0 +1,161 @@ +#!/usr/bin/env bash +# +# Tests for the devtools scripts. +# +# Each case drives a script into a specific failure mode and asserts it is +# detected. A script that only ever reports PASS is worse than no script, so the +# negative cases are the point — the positive ones just confirm it isn't +# failing everything. + +set -uo pipefail + +BIN=$(cd "$(dirname "${BASH_SOURCE[0]}")/../bin" && pwd) +pass=0; fail=0 +tmproot=$(mktemp -d); trap 'rm -rf "$tmproot"' EXIT + +ok() { printf ' PASS %s\n' "$1"; pass=$((pass+1)); } +bad() { printf ' FAIL %s\n -> %s\n' "$1" "$2"; fail=$((fail+1)); } + +# Builds a fake project dir containing a gradlew that behaves as told. +# $1 dir, $2 exit code for gradlew, $3 optional: "write-xml <tests> <failures>" +make_project() { + local d=$1 code=$2; shift 2 + mkdir -p "$d" + cat > "$d/gradlew" <<EOF +#!/usr/bin/env bash +echo "BUILD SUCCESSFUL in 1s" # deliberately lies when exit code is nonzero +$* +exit $code +EOF + chmod +x "$d/gradlew" +} + +xml_writer() { + local dir=$1 tests=$2 failures=$3 + printf 'mkdir -p "%s"; cat > "%s/TEST-Fake.xml" <<XML\n<testsuite name="Fake" tests="%s" failures="%s" errors="0" skipped="0"></testsuite>\nXML\n' \ + "$dir" "$dir" "$tests" "$failures" +} + +printf '=== gradle-verify ===\n' + +# 1. Honest failure: gradlew exits nonzero but prints BUILD SUCCESSFUL. +p="$tmproot/p1"; make_project "$p" 1 +out=$("$BIN/gradle-verify" "$p" someTask 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "result : FAIL"; then + ok "reports FAIL on nonzero exit even when output says BUILD SUCCESSFUL" +else + bad "grep-proof exit code" "rc=$rc out=$(printf '%s' "$out" | tail -2 | tr '\n' ' ')" +fi + +# 2. Success path. +p="$tmproot/p2"; make_project "$p" 0 +out=$("$BIN/gradle-verify" "$p" someTask 2>&1); rc=$? +if [ $rc -eq 0 ] && printf '%s' "$out" | grep -q "result : PASS"; then + ok "reports PASS on zero exit" +else + bad "success path" "rc=$rc" +fi + +# 3. Nonexistent project directory must be a hard error, not a silent fallback +# to the current directory. This is the cwd-drift bug. +out=$("$BIN/gradle-verify" "$tmproot/does-not-exist" someTask 2>&1); rc=$? +if [ $rc -eq 2 ] && printf '%s' "$out" | grep -qi "does not exist"; then + ok "refuses a missing project dir instead of falling back to cwd" +else + bad "missing dir is fatal" "rc=$rc out=$out" +fi + +# 4. A directory with no gradlew is also fatal. +mkdir -p "$tmproot/p4" +out=$("$BIN/gradle-verify" "$tmproot/p4" someTask 2>&1); rc=$? +if [ $rc -eq 2 ] && printf '%s' "$out" | grep -qi "no executable gradlew"; then + ok "refuses a dir with no gradlew" +else + bad "missing gradlew is fatal" "rc=$rc out=$out" +fi + +# 5. The project used must be the one named, regardless of cwd. +p="$tmproot/p5"; make_project "$p" 0 +out=$(cd /tmp && "$BIN/gradle-verify" "$p" someTask 2>&1); rc=$? +if printf '%s' "$out" | grep -q "project : $p"; then + ok "names the resolved absolute project dir, independent of cwd" +else + bad "cwd independence" "$out" +fi + +printf '=== gradle-test ===\n' + +# 6. Zero tests must fail even though the build succeeded. +p="$tmproot/t1"; results="$p/build/test-results/test" +make_project "$p" 0 "$(xml_writer "$results" 0 0)" +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "zero tests executed"; then + ok "fails when a green build executed no tests" +else + bad "zero-test detection" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +# 7. Real failures are reported. +p="$tmproot/t2"; results="$p/build/test-results/test" +make_project "$p" 1 "$(xml_writer "$results" 10 3)" +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "failures: 3"; then + ok "reports failure counts parsed from XML" +else + bad "failure counts" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +# 8. Passing suite passes. +p="$tmproot/t3"; results="$p/build/test-results/test" +make_project "$p" 0 "$(xml_writer "$results" 42 0)" +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -eq 0 ] && printf '%s' "$out" | grep -q "result : PASS (42 tests"; then + ok "passes and reports the parsed count" +else + bad "pass path" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +# 9. Stale results: XML exists but predates the run (the UP-TO-DATE case). +p="$tmproot/t4"; results="$p/build/test-results/test" +make_project "$p" 0 # gradlew writes nothing this time +mkdir -p "$results" +cat > "$results/TEST-Old.xml" <<'XML' +<testsuite name="Old" tests="99" failures="0" errors="0" skipped="0"></testsuite> +XML +touch -d '2020-01-01 00:00:00' "$results/TEST-Old.xml" +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "predates this run"; then + ok "refuses to report stale results as current" +else + bad "stale detection" "rc=$rc out=$(printf '%s' "$out" | tail -4 | tr '\n' ' ')" +fi + +# 10. ...unless explicitly allowed. +out=$("$BIN/gradle-test" "$p" test --allow-stale 2>&1); rc=$? +if [ $rc -eq 0 ] && printf '%s' "$out" | grep -q "99"; then + ok "--allow-stale opts in to counting old results" +else + bad "allow-stale" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +# 11. No XML at all. +p="$tmproot/t5"; make_project "$p" 0 +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "no JUnit XML found"; then + ok "fails when no results were produced at all" +else + bad "no-xml detection" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +printf '=== android-launch-check ===\n' + +# 12. Argument validation without a device. +out=$("$BIN/android-launch-check" 2>&1); rc=$? +if [ $rc -eq 2 ] && printf '%s' "$out" | grep -qi "usage"; then + ok "requires a package argument" +else + bad "usage check" "rc=$rc out=$out" +fi + +printf '\n=== summary: %d passed, %d failed ===\n' "$pass" "$fail" +[ "$fail" -eq 0 ] |
