summaryrefslogtreecommitdiff
path: root/feedback_verify_from_exit_codes_not_output.md
diff options
context:
space:
mode:
Diffstat (limited to 'feedback_verify_from_exit_codes_not_output.md')
-rw-r--r--feedback_verify_from_exit_codes_not_output.md57
1 files changed, 57 insertions, 0 deletions
diff --git a/feedback_verify_from_exit_codes_not_output.md b/feedback_verify_from_exit_codes_not_output.md
new file mode 100644
index 0000000..ead8323
--- /dev/null
+++ b/feedback_verify_from_exit_codes_not_output.md
@@ -0,0 +1,57 @@
+---
+name: feedback-verify-from-exit-codes-not-output
+description: "Never conclude a build/test/launch result from grepped console text — use /workspace/devtools wrappers, which report from exit codes and structured artifacts"
+metadata:
+ node_type: memory
+ type: feedback
+ originSessionId: 08c1f89c-4d24-4fb2-9488-0fe155872388
+ modified: 2026-09-18T21:05:18.382Z
+---
+
+**Never report a build, test, or launch result based on grepped console output.**
+Use the shared wrappers, invoked by absolute path:
+
+```
+/workspace/devtools/bin/gradle-verify <project-dir> <task...>
+/workspace/devtools/bin/gradle-test <project-dir> [task] -- --rerun-tasks
+/workspace/devtools/bin/android-launch-check <package> [--apk <path>] [--settle <s>]
+```
+
+**Why:** in one session (2026-09-18) I reported three results that were about
+something other than what I believed I was testing, and Peter noticed:
+
+1. A failed `cd` in a chained command left the shell in the previous directory;
+ the `BUILD SUCCESSFUL` I then reported was for a different project tree.
+2. `adb shell monkey` + a single `pidof` sample reported a healthy app as crashed
+ and a crashed app as healthy, on the same emulator minutes apart.
+3. `adb install` refused a filename (not ending `.apk`) and said so on stdout,
+ which I had redirected to `/dev/null`; the previous build stayed installed and
+ was what actually got tested.
+
+`gradle-test` caught a fourth immediately on first real use: nav reporting
+`UP-TO-DATE` with 33 stale result files, which read naively as "241 tests pass"
+without a single test having run.
+
+**Shell traps that cause this class of error** (all encoded in the wrappers, all
+worth avoiding in ad-hoc commands too):
+
+- `cmd | tail; echo $?` reports `tail`'s status. Redirect to a file and check
+ `$?`, or use `${PIPESTATUS[0]}`.
+- `... | grep -q` under `set -o pipefail` SIGPIPEs the upstream command when grep
+ exits early on a match — the pipeline reports failure *because* it matched. Do
+ membership tests in pure shell (`case`).
+- `set -e` plus `pid=$(pidof x)` aborts the script exactly when the process is
+ gone, i.e. when you are trying to detect a crash. Use `|| true` on any
+ substitution whose non-zero status is meaningful data.
+- Gradle `UP-TO-DATE` means results on disk describe older code. Pass
+ `-- --rerun-tasks` whenever the number matters.
+- A green build with zero tests executed is not a passing suite.
+
+**How to apply:** when stating a result to Peter, the number must come from a
+fresh parsed artifact, not from console text. If a claim can't be backed that
+way, say so rather than asserting it. He does not read the code, so an
+unverified claim is one he cannot catch.
+
+See `/workspace/devtools/README.md` for the full trap table. Referenced from
+nav's and feedreader's `CLAUDE.md`. Related: [[project_feedreader_apk_publishing]]
+(the `am start` vs `monkey` detail), [[feedback_verify_before_asserting_root_cause]].