From 36a82d934366cc2ddbd92ee3885b1c547eeb8244 Mon Sep 17 00:00:00 2001 From: Peter Stone Date: Sun, 20 Sep 2026 00:36:17 +0000 Subject: Fold the DB-ping health check in; delete the watchdog that never ran MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit scripts/health-watchdog.sh was written after the 2026-08-04 incident, where a liveness stub stayed green for three days while every database-touching request path was wedged. Its header says "Meant to run from cron every few minutes". It was never scheduled — not in any unit, not in any crontab — so it has been dead code since August, offering coverage it did not provide. Its one real idea is worth keeping: /health pings the database and returns 503 when that fails, which the gateway route checks cannot see, because `/` answers 303 to the login page without going near the database. check_app_health now does that poll (two attempts, five seconds apart, so a single blip is not an outage), verify-deploy asserts it, and health-check recovers from it by restarting the main service — the action the old watchdog would have taken. The dead script is removed rather than left to imply cover. Tests now 11, including a stub server returning 503 to prove a listening-but- wedged app is caught. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B --- scripts/verify-deploy | 3 +++ 1 file changed, 3 insertions(+) (limited to 'scripts/verify-deploy') diff --git a/scripts/verify-deploy b/scripts/verify-deploy index 712c26d..db3b1d8 100755 --- a/scripts/verify-deploy +++ b/scripts/verify-deploy @@ -32,6 +32,9 @@ check_services || rc=1 echo "==> Verifying gateway routes..." check_routes || rc=1 +echo "==> Verifying app health (database reachable)..." +check_app_health || rc=1 + if [ -n "$expected_build" ]; then echo "==> Verifying deployed build..." check_build_stamp "$expected_build" || rc=1 -- cgit v1.2.3