diff options
| -rw-r--r-- | .gitignore | 1 | ||||
| -rw-r--r-- | README.md | 88 | ||||
| -rwxr-xr-x | bin/android-launch-check | 159 | ||||
| -rwxr-xr-x | bin/gradle-test | 140 | ||||
| -rwxr-xr-x | bin/gradle-verify | 63 | ||||
| -rwxr-xr-x | test/run-tests.sh | 161 |
6 files changed, 612 insertions, 0 deletions
diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..f47cb20 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +*.out diff --git a/README.md b/README.md new file mode 100644 index 0000000..e3b96c6 --- /dev/null +++ b/README.md @@ -0,0 +1,88 @@ +# devtools + +Small, shared wrappers around Gradle and adb whose only job is to make a build or +test result **impossible to misread**. + +These exist because of a specific, repeated failure: concluding that something +passed or failed by reading console text, when the text was about a different +project, a stale result, or a command that never ran. Every script here reports +from exit codes and structured artifacts instead, and fails loudly rather than +defaulting to something plausible. + +Invoke by absolute path — `/workspace/devtools/bin/<script>`. Nothing needs to be +on `PATH`, which is itself one less piece of ambient state to get wrong. + +--- + +## The rule these encode + +> **Never conclude from grepped stdout. Conclude from exit codes and structured +> artifacts.** + +Corollaries, each learned the hard way: + +| Trap | What it looks like | Guard | +| --- | --- | --- | +| cwd drift | `cd x && cp y .` fails; the next line runs Gradle in the old directory and prints `BUILD SUCCESSFUL` about the wrong project | `gradle-verify` takes the project dir as an argument, resolves it absolutely, and passes `-p`. A missing dir is fatal, never a fallback. | +| Stale test results | Gradle reports `UP-TO-DATE`, runs nothing, and the XML on disk describes older code | `gradle-test` compares result mtimes against a marker written at run start and refuses stale numbers | +| Zero tests | A test source set that won't compile leaves a green build with nothing executed | `gradle-test` fails when the parsed test count is 0 | +| `cmd \| tail; echo $?` | Reports `tail`'s status, not the command's | Redirect to a file, then check `$?`. Or use `PIPESTATUS[0]`. | +| `... \| grep -q` under `pipefail` | `grep -q` exits on first match and SIGPIPEs upstream; the pipeline reports failure despite matching | Do membership tests in pure shell (`case`), not pipelines | +| `set -e` + a command that legitimately fails | `pid=$(pidof x)` aborts the whole script when the process is gone — i.e. exactly when you are trying to detect a crash | `|| true` on any substitution whose non-zero status is meaningful data | +| Discarded command output | `adb install ... >/dev/null` hides "filename doesn't end .apk"; the old build stays installed and is silently what gets tested | `android-launch-check --apk` checks the install output for `Success` | +| `monkey` as a launch check | Reports a healthy app dead and a dead app healthy; lands on whatever activity is foreground | `android-launch-check` uses `am start -W` against a resolved activity and samples the pid repeatedly | + +--- + +## Scripts + +### `gradle-verify <project-dir> <task...> [-- <gradle args>]` + +Runs Gradle against an explicitly named project. Exits with Gradle's exit code. +Prints the absolute project directory actually used, so the output is +self-describing. + +``` +/workspace/devtools/bin/gradle-verify /workspace/nav/android-app compileDebugKotlin +``` + +### `gradle-test <project-dir> [task] [--allow-stale] [-- <gradle args>]` + +Runs the test task, then reports counts parsed from the JUnit XML. Auto-detects +`testDebugUnitTest` for Android projects and `test` otherwise. + +Fails on: non-zero Gradle exit, any failure or error in the XML, **zero tests +executed**, **no XML at all**, or **results that predate this run**. + +``` +/workspace/devtools/bin/gradle-test /workspace/nav/android-app -- --rerun-tasks +``` + +`--rerun-tasks` is usually what you want when the answer matters — otherwise +Gradle may skip the task and the script will (correctly) refuse the stale +results. + +### `android-launch-check <package> [--apk <path>] [--serial <s>] [--settle <secs>]` + +Answers "does this build actually start and stay up", and shows the exception if +not. Optionally installs an APK first, verifying the install genuinely succeeded. + +Exit codes: `0` launched and stayed up · `1` crashed or died · `2` operator error +(no device, package not installed, unusable filename). + +``` +/workspace/devtools/bin/android-launch-check org.terst.feedreader \ + --apk /path/to/app-debug.apk --settle 15 +``` + +--- + +## Tests + +``` +/workspace/devtools/test/run-tests.sh +``` + +12 cases, mostly negative — each drives a script into one failure mode and +asserts it is caught. A wrapper that only ever reports PASS is worse than no +wrapper, so the negative cases are the point. diff --git a/bin/android-launch-check b/bin/android-launch-check new file mode 100755 index 0000000..00dfcfd --- /dev/null +++ b/bin/android-launch-check @@ -0,0 +1,159 @@ +#!/usr/bin/env bash +# +# Determine whether an Android app actually launches and stays up, and surface +# the exception if it does not. +# +# Why this exists: `adb shell monkey -p <pkg> -c android.intent.category.LAUNCHER 1` +# followed by polling `pidof` is not a reliable check. In practice it reported a +# healthy app as dead and a dead app as healthy, in the same session, on the same +# emulator — monkey's launch can land on whatever activity happens to be in the +# foreground, and a single `pidof` sample races app startup and restart. +# +# This script instead: +# * resolves the launcher activity from the package manager rather than guessing +# * starts a continuous logcat capture BEFORE launching, so the crash window is +# never missed (the `-b crash` buffer was empty even for real FATAL crashes) +# * launches with `am start -W` and checks its reported status +# * samples the pid repeatedly over a settle period, so a process that starts +# and then dies is reported as a failure, not a success +# * greps the captured log for FATAL EXCEPTION and prints the stack +# +# Usage: +# android-launch-check <package> [--serial <adb-serial>] [--settle <seconds>] +# +# Exit: 0 = launched and stayed up with no fatal exception, 1 = did not. + +set -euo pipefail + +die() { printf '%s: %s\n' "$(basename "$0")" "$*" >&2; exit 2; } + +[ $# -ge 1 ] || die "usage: $(basename "$0") <package> [--serial <s>] [--settle <secs>]" + +pkg=$1; shift +serial="" +settle=20 +apk="" +while [ $# -gt 0 ]; do + case "$1" in + --serial) serial=${2:-}; shift 2 ;; + --settle) settle=${2:-20}; shift 2 ;; + --apk) apk=${2:-}; shift 2 ;; + *) die "unknown argument: $1" ;; + esac +done + +adb_bin=$(command -v adb || echo /opt/android-sdk/platform-tools/adb) +[ -x "$adb_bin" ] || die "adb not found" +ADB=("$adb_bin") +[ -n "$serial" ] && ADB+=(-s "$serial") + +devices=$("${ADB[@]}" devices | awk 'NR>1 && $2=="device" {print $1}') +[ -n "$devices" ] || die "no adb device in 'device' state" +[ "$(printf '%s\n' "$devices" | wc -l)" -eq 1 ] || [ -n "$serial" ] \ + || die "multiple devices attached; pass --serial. Found: $(echo $devices)" + +printf '=== android-launch-check ===\n' +printf 'package : %s\n' "$pkg" +printf 'device : %s\n' "$(echo $devices | tr '\n' ' ')" + +if [ -n "$apk" ]; then + [ -f "$apk" ] || die "apk not found: $apk" + # adb rejects any filename not ending .apk/.apex, and prints that refusal to + # stdout while still exiting 0 in some versions — so check the text as well as + # the status. Discarding this output is how a "reinstall" can silently not + # happen and leave the previous build under test. + case "$apk" in + *.apk|*.apex) : ;; + *) die "adb will refuse this filename (must end .apk/.apex): $apk" ;; + esac + printf 'apk : %s\n' "$apk" + install_out=$("${ADB[@]}" install -r "$apk" 2>&1 | tr -d '\r' || true) + if ! printf '%s' "$install_out" | grep -q '^Success$'; then + printf 'result : FAIL — install did not succeed\n' + printf '%s\n' "$install_out" | sed 's/^/ /' + exit 1 + fi + printf 'install : Success\n' +fi + +# Membership test done in pure shell, not `... | grep -q`. Under `set -o pipefail` +# a `grep -q` that exits on its first match can SIGPIPE the upstream command, and +# the pipeline then reports failure even though the match succeeded — which made +# this check report an installed package as missing. +pkg_list=$("${ADB[@]}" shell pm list packages 2>/dev/null | tr -d '\r' || true) +case $'\n'"$pkg_list"$'\n' in + *$'\n'"package:$pkg"$'\n'*) : ;; + *) die "package not installed: $pkg" ;; +esac + +# Resolve rather than assume. Guessing ".MainActivity" when the manifest says +# ".ui.MainActivity" produces "Activity class does not exist", which is an +# operator error that can easily be misread as an app crash. +activity=$("${ADB[@]}" shell cmd package resolve-activity --brief "$pkg" 2>/dev/null \ + | tr -d '\r' | tail -1) +case "$activity" in + "$pkg"/*) : ;; + *) die "could not resolve launcher activity for $pkg (got: '$activity')" ;; +esac +printf 'activity: %s\n' "$activity" + +logfile=$(mktemp); trap 'rm -f "$logfile"' EXIT + +"${ADB[@]}" shell am force-stop "$pkg" >/dev/null 2>&1 || true +sleep 2 +"${ADB[@]}" logcat -c >/dev/null 2>&1 || true + +# Capture starts first and runs across the whole window. +"${ADB[@]}" logcat -v threadtime > "$logfile" 2>&1 & +logcat_pid=$! +trap 'kill "$logcat_pid" 2>/dev/null || true; rm -f "$logfile"' EXIT +sleep 2 + +start_out=$("${ADB[@]}" shell am start -W -n "$activity" 2>&1 | tr -d '\r') +start_status=$(printf '%s\n' "$start_out" | awk -F': *' '/^Status:/ {print $2}') +printf 'am start: %s\n' "${start_status:-<none>}" + +# Sample repeatedly. One sample cannot distinguish "not started yet" from +# "started and already died". +alive_samples=0; total_samples=0; last_pid="" +interval=5 +elapsed=0 +while [ "$elapsed" -lt "$settle" ]; do + sleep "$interval" + elapsed=$((elapsed + interval)) + # `|| true` is load-bearing: pidof exits non-zero when the process is gone, + # which under `set -e` would abort this script at exactly the moment it is + # supposed to be detecting a crash. + pid=$("${ADB[@]}" shell pidof "$pkg" 2>/dev/null | tr -d '\r' | awk '{print $1}' || true) + total_samples=$((total_samples + 1)) + if [ -n "$pid" ]; then alive_samples=$((alive_samples + 1)); last_pid=$pid; fi + printf ' t=%-3ss pid=%s\n' "$elapsed" "${pid:-<none>}" +done + +sleep 1 +kill "$logcat_pid" 2>/dev/null || true +wait "$logcat_pid" 2>/dev/null || true + +fatal_count=$(grep -c "FATAL EXCEPTION" "$logfile" 2>/dev/null || true) +fatal_count=${fatal_count:-0} + +printf 'samples : %d/%d alive\n' "$alive_samples" "$total_samples" +printf 'fatal : %s\n' "$fatal_count" + +if [ "$fatal_count" -gt 0 ]; then + printf '\n--- first fatal exception ---\n' + grep -A 25 -m 1 "FATAL EXCEPTION" "$logfile" | sed 's/^/ /' + printf '\nresult : FAIL — app crashed\n' + exit 1 +fi + +if [ "$alive_samples" -ne "$total_samples" ]; then + printf '\nresult : FAIL — process was not alive for the whole settle window\n' + printf 'hint : started but died, or never started. No FATAL in logcat, so\n' + printf ' check for a native crash or an early System.exit.\n' + exit 1 +fi + +printf 'pid : %s\n' "$last_pid" +printf 'result : PASS — launched and stayed up %ss with no fatal exception\n' "$settle" +exit 0 diff --git a/bin/gradle-test b/bin/gradle-test new file mode 100755 index 0000000..5b059d7 --- /dev/null +++ b/bin/gradle-test @@ -0,0 +1,140 @@ +#!/usr/bin/env bash +# +# Run a Gradle test task and report counts parsed from the JUnit XML it wrote, +# not from console text. +# +# Catches three failure modes that all look like success from the console: +# +# 1. Zero tests ran. A unit-test source set that fails to compile, or a filter +# that matches nothing, can leave the build green with nothing executed. +# "BUILD SUCCESSFUL" and "0 tests" are the same line of output. +# 2. Stale results. If Gradle decides the test task is UP-TO-DATE it does not +# re-run anything, and the XML on disk describes an older version of the +# code. Reading those numbers as current is how a passing suite gets +# reported for code that was never tested. +# 3. Results from the wrong project, via cwd drift — handled by gradle-verify. +# +# Usage: +# gradle-test <project-dir> [test-task] [--allow-stale] [-- <gradle args>] +# +# Default task: testDebugUnitTest for Android projects, test otherwise. + +set -euo pipefail + +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +die() { printf '%s: %s\n' "$(basename "$0")" "$*" >&2; exit 2; } + +[ $# -ge 1 ] || die "usage: $(basename "$0") <project-dir> [test-task] [--allow-stale] [-- <gradle args>]" + +raw_dir=$1; shift +project_dir=$(cd "$raw_dir" 2>/dev/null && pwd) || die "no such project dir: $raw_dir" + +task="" +allow_stale=0 +extra=() +seen_sep=0 +for arg in "$@"; do + if [ "$arg" = "--" ]; then seen_sep=1; continue; fi + if [ $seen_sep -eq 1 ]; then extra+=("$arg"); continue; fi + case "$arg" in + --allow-stale) allow_stale=1 ;; + -*) extra+=("$arg") ;; + *) task=$arg ;; + esac +done + +if [ -z "$task" ]; then + if grep -rqs "com.android.application\|com.android.library" \ + "$project_dir"/build.gradle "$project_dir"/build.gradle.kts \ + "$project_dir"/*/build.gradle "$project_dir"/*/build.gradle.kts 2>/dev/null; then + task=testDebugUnitTest + else + task=test + fi +fi + +# Marker file: anything written at or after this is from the run we just did. +# Using a file rather than a clock reading avoids any filesystem/shell timestamp +# granularity mismatch. +marker=$(mktemp) +trap 'rm -f "$marker"' EXIT +sleep 1 # ensure a strictly-earlier mtime than anything the build writes + +set +e +"$here/gradle-verify" "$project_dir" "$task" ${extra[@]+-- "${extra[@]}"} +gradle_status=$? +set -e + +printf '\n=== gradle-test results ===\n' + +mapfile -t all_xml < <(find "$project_dir" -path '*/test-results/*' -name 'TEST-*.xml' 2>/dev/null | sort) +if [ ${#all_xml[@]} -eq 0 ]; then + printf 'result : FAIL — no JUnit XML found anywhere under %s\n' "$project_dir" + printf 'hint : the test task may not have run, or writes results elsewhere\n' + exit 1 +fi + +fresh=(); stale=() +for f in "${all_xml[@]}"; do + if [ "$f" -nt "$marker" ]; then fresh+=("$f"); else stale+=("$f"); fi +done + +printf 'task : %s\n' "$task" +printf 'xml : %d file(s) total, %d fresh, %d stale\n' \ + "${#all_xml[@]}" "${#fresh[@]}" "${#stale[@]}" + +if [ ${#fresh[@]} -eq 0 ]; then + if [ $allow_stale -eq 1 ]; then + printf 'note : no fresh results; counting stale ones because --allow-stale\n' + fresh=("${all_xml[@]}") + else + printf 'result : FAIL — every result file predates this run\n' + printf 'hint : Gradle treated %s as UP-TO-DATE, so these numbers describe\n' "$task" + printf ' older code. Re-run with: -- --rerun-tasks\n' + exit 1 + fi +fi + +counts=$(python3 - "${fresh[@]}" <<'PY' +import sys, xml.etree.ElementTree as ET +tests = failures = errors = skipped = 0 +bad = [] +for p in sys.argv[1:]: + try: + r = ET.parse(p).getroot() + except Exception as e: + bad.append(f"{p}: unparseable ({e})") + continue + tests += int(r.get('tests', 0)) + failures += int(r.get('failures', 0)) + errors += int(r.get('errors', 0)) + skipped += int(r.get('skipped', 0)) +print(f"{tests} {failures} {errors} {skipped}") +for b in bad: + print("UNPARSEABLE " + b, file=sys.stderr) +PY +) +read -r tests failures errors skipped <<<"$counts" + +printf 'tests : %s\n' "$tests" +printf 'failures: %s\n' "$failures" +printf 'errors : %s\n' "$errors" +printf 'skipped : %s\n' "$skipped" + +rc=0 +[ "$gradle_status" -ne 0 ] && rc=1 +[ "$failures" -gt 0 ] && rc=1 +[ "$errors" -gt 0 ] && rc=1 +if [ "$tests" -eq 0 ]; then + printf 'result : FAIL — zero tests executed\n' + printf 'hint : a green build with no tests is not a passing suite\n' + exit 1 +fi + +if [ $rc -eq 0 ]; then + printf 'result : PASS (%s tests, gradle exit %d)\n' "$tests" "$gradle_status" +else + printf 'result : FAIL (gradle exit %d, %s failures, %s errors)\n' \ + "$gradle_status" "$failures" "$errors" +fi +exit $rc diff --git a/bin/gradle-verify b/bin/gradle-verify new file mode 100755 index 0000000..73fb69d --- /dev/null +++ b/bin/gradle-verify @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# +# Run Gradle tasks against an explicitly-named project and report the result from +# the exit code, never from the console text. +# +# Why this exists: reading "BUILD SUCCESSFUL" out of piped Gradle output is not a +# reliable signal. A shell chain like `cd dir && cp f . ` followed on the next +# line by `./gradlew test` will happily run Gradle in whatever directory the +# shell was already in if the cd failed, and the success line it prints is then +# about a different project entirely. That failure mode is silent and reads +# exactly like success. +# +# Usage: +# gradle-verify <project-dir> <gradle-task...> [-- <extra gradle args>] +# +# Exits with Gradle's own exit code. Prints a summary block that names the +# absolute project directory actually used. + +set -euo pipefail + +die() { printf '%s: %s\n' "$(basename "$0")" "$*" >&2; exit 2; } + +[ $# -ge 2 ] || die "usage: $(basename "$0") <project-dir> <gradle-task...> [-- <gradle args>]" + +raw_dir=$1; shift + +# Resolve to an absolute path up front and fail loudly if it is not what we +# expect. Every later step uses this resolved value, so there is nothing for an +# ambient cwd to influence. +project_dir=$(cd "$raw_dir" 2>/dev/null && pwd) \ + || die "project dir does not exist or is not readable: $raw_dir" +[ -x "$project_dir/gradlew" ] \ + || die "no executable gradlew in: $project_dir" + +tasks=() +extra=() +seen_sep=0 +for arg in "$@"; do + if [ "$arg" = "--" ]; then seen_sep=1; continue; fi + if [ $seen_sep -eq 1 ]; then extra+=("$arg"); else tasks+=("$arg"); fi +done +[ ${#tasks[@]} -ge 1 ] || die "no gradle tasks given" + +printf '=== gradle-verify ===\n' +printf 'project : %s\n' "$project_dir" +printf 'tasks : %s\n' "${tasks[*]}" +[ ${#extra[@]} -gt 0 ] && printf 'args : %s\n' "${extra[*]}" + +# -p pins the project directory regardless of cwd. Note gradlew is invoked by +# absolute path too, so we are not depending on PATH or on cwd for the wrapper +# either. +set +e +"$project_dir/gradlew" -p "$project_dir" "${tasks[@]}" ${extra[@]+"${extra[@]}"} +status=$? +set -e + +printf 'exit : %d\n' "$status" +if [ $status -eq 0 ]; then + printf 'result : PASS\n' +else + printf 'result : FAIL\n' +fi +exit $status diff --git a/test/run-tests.sh b/test/run-tests.sh new file mode 100755 index 0000000..3a8117d --- /dev/null +++ b/test/run-tests.sh @@ -0,0 +1,161 @@ +#!/usr/bin/env bash +# +# Tests for the devtools scripts. +# +# Each case drives a script into a specific failure mode and asserts it is +# detected. A script that only ever reports PASS is worse than no script, so the +# negative cases are the point — the positive ones just confirm it isn't +# failing everything. + +set -uo pipefail + +BIN=$(cd "$(dirname "${BASH_SOURCE[0]}")/../bin" && pwd) +pass=0; fail=0 +tmproot=$(mktemp -d); trap 'rm -rf "$tmproot"' EXIT + +ok() { printf ' PASS %s\n' "$1"; pass=$((pass+1)); } +bad() { printf ' FAIL %s\n -> %s\n' "$1" "$2"; fail=$((fail+1)); } + +# Builds a fake project dir containing a gradlew that behaves as told. +# $1 dir, $2 exit code for gradlew, $3 optional: "write-xml <tests> <failures>" +make_project() { + local d=$1 code=$2; shift 2 + mkdir -p "$d" + cat > "$d/gradlew" <<EOF +#!/usr/bin/env bash +echo "BUILD SUCCESSFUL in 1s" # deliberately lies when exit code is nonzero +$* +exit $code +EOF + chmod +x "$d/gradlew" +} + +xml_writer() { + local dir=$1 tests=$2 failures=$3 + printf 'mkdir -p "%s"; cat > "%s/TEST-Fake.xml" <<XML\n<testsuite name="Fake" tests="%s" failures="%s" errors="0" skipped="0"></testsuite>\nXML\n' \ + "$dir" "$dir" "$tests" "$failures" +} + +printf '=== gradle-verify ===\n' + +# 1. Honest failure: gradlew exits nonzero but prints BUILD SUCCESSFUL. +p="$tmproot/p1"; make_project "$p" 1 +out=$("$BIN/gradle-verify" "$p" someTask 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "result : FAIL"; then + ok "reports FAIL on nonzero exit even when output says BUILD SUCCESSFUL" +else + bad "grep-proof exit code" "rc=$rc out=$(printf '%s' "$out" | tail -2 | tr '\n' ' ')" +fi + +# 2. Success path. +p="$tmproot/p2"; make_project "$p" 0 +out=$("$BIN/gradle-verify" "$p" someTask 2>&1); rc=$? +if [ $rc -eq 0 ] && printf '%s' "$out" | grep -q "result : PASS"; then + ok "reports PASS on zero exit" +else + bad "success path" "rc=$rc" +fi + +# 3. Nonexistent project directory must be a hard error, not a silent fallback +# to the current directory. This is the cwd-drift bug. +out=$("$BIN/gradle-verify" "$tmproot/does-not-exist" someTask 2>&1); rc=$? +if [ $rc -eq 2 ] && printf '%s' "$out" | grep -qi "does not exist"; then + ok "refuses a missing project dir instead of falling back to cwd" +else + bad "missing dir is fatal" "rc=$rc out=$out" +fi + +# 4. A directory with no gradlew is also fatal. +mkdir -p "$tmproot/p4" +out=$("$BIN/gradle-verify" "$tmproot/p4" someTask 2>&1); rc=$? +if [ $rc -eq 2 ] && printf '%s' "$out" | grep -qi "no executable gradlew"; then + ok "refuses a dir with no gradlew" +else + bad "missing gradlew is fatal" "rc=$rc out=$out" +fi + +# 5. The project used must be the one named, regardless of cwd. +p="$tmproot/p5"; make_project "$p" 0 +out=$(cd /tmp && "$BIN/gradle-verify" "$p" someTask 2>&1); rc=$? +if printf '%s' "$out" | grep -q "project : $p"; then + ok "names the resolved absolute project dir, independent of cwd" +else + bad "cwd independence" "$out" +fi + +printf '=== gradle-test ===\n' + +# 6. Zero tests must fail even though the build succeeded. +p="$tmproot/t1"; results="$p/build/test-results/test" +make_project "$p" 0 "$(xml_writer "$results" 0 0)" +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "zero tests executed"; then + ok "fails when a green build executed no tests" +else + bad "zero-test detection" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +# 7. Real failures are reported. +p="$tmproot/t2"; results="$p/build/test-results/test" +make_project "$p" 1 "$(xml_writer "$results" 10 3)" +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "failures: 3"; then + ok "reports failure counts parsed from XML" +else + bad "failure counts" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +# 8. Passing suite passes. +p="$tmproot/t3"; results="$p/build/test-results/test" +make_project "$p" 0 "$(xml_writer "$results" 42 0)" +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -eq 0 ] && printf '%s' "$out" | grep -q "result : PASS (42 tests"; then + ok "passes and reports the parsed count" +else + bad "pass path" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +# 9. Stale results: XML exists but predates the run (the UP-TO-DATE case). +p="$tmproot/t4"; results="$p/build/test-results/test" +make_project "$p" 0 # gradlew writes nothing this time +mkdir -p "$results" +cat > "$results/TEST-Old.xml" <<'XML' +<testsuite name="Old" tests="99" failures="0" errors="0" skipped="0"></testsuite> +XML +touch -d '2020-01-01 00:00:00' "$results/TEST-Old.xml" +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "predates this run"; then + ok "refuses to report stale results as current" +else + bad "stale detection" "rc=$rc out=$(printf '%s' "$out" | tail -4 | tr '\n' ' ')" +fi + +# 10. ...unless explicitly allowed. +out=$("$BIN/gradle-test" "$p" test --allow-stale 2>&1); rc=$? +if [ $rc -eq 0 ] && printf '%s' "$out" | grep -q "99"; then + ok "--allow-stale opts in to counting old results" +else + bad "allow-stale" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +# 11. No XML at all. +p="$tmproot/t5"; make_project "$p" 0 +out=$("$BIN/gradle-test" "$p" test 2>&1); rc=$? +if [ $rc -ne 0 ] && printf '%s' "$out" | grep -q "no JUnit XML found"; then + ok "fails when no results were produced at all" +else + bad "no-xml detection" "rc=$rc out=$(printf '%s' "$out" | tail -3 | tr '\n' ' ')" +fi + +printf '=== android-launch-check ===\n' + +# 12. Argument validation without a device. +out=$("$BIN/android-launch-check" 2>&1); rc=$? +if [ $rc -eq 2 ] && printf '%s' "$out" | grep -qi "usage"; then + ok "requires a package argument" +else + bad "usage check" "rc=$rc out=$out" +fi + +printf '\n=== summary: %d passed, %d failed ===\n' "$pass" "$fail" +[ "$fail" -eq 0 ] |