doot.git/scripts, branch master

Fold the DB-ping health check in; delete the watchdog that never ran

2026-09-20T00:36:17+00:00

scripts/health-watchdog.sh was written after the 2026-08-04 incident, where a liveness stub stayed green for three days while every database-touching request path was wedged. Its header says "Meant to run from cron every few minutes". It was never scheduled — not in any unit, not in any crontab — so it has been dead code since August, offering coverage it did not provide. Its one real idea is worth keeping: /health pings the database and returns 503 when that fails, which the gateway route checks cannot see, because `/` answers 303 to the login page without going near the database. check_app_health now does that poll (two attempts, five seconds apart, so a single blip is not an outage), verify-deploy asserts it, and health-check recovers from it by restarting the main service — the action the old watchdog would have taken. The dead script is removed rather than left to imply cover. Tests now 11, including a stub server returning 503 to prove a listening-but- wedged app is caught. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Verify deploys, and add a self-recovering health check

2026-09-20T00:34:23+00:00

Deploy used to end with `systemctl status`, which a human skims and a script ignores: it exits zero whether or not anything works, and says nothing about the sub-services behind the gateway. A deploy could therefore report success while /scout/ served 502s — which is exactly what happened, undetected, for three and a half weeks. scripts/deploy now: * syncs scripts/ to ${SITE_DIR}/app-scripts/ so the timer runs the deployed version from a stable path * restarts sub-services that are not active. They are separate units and a doot deploy does not touch them, so one sitting in `failed` after exhausting its restart limit stays down through any number of green deploys. * calls scripts/verify-deploy, which asserts and exits non-zero scripts/verify-deploy asserts all five binaries are executable, all four units are active, all four gateway routes answer without a 5xx, and the running build stamp matches the commit just built. scripts/health-check runs the same assertions on a 5-minute timer and recovers what it can: chmod +x a binary that lost its execute bit, reset-failed and start a unit that gave up. There is no mail/ntfy/apprise on this host, so alerting is a greppable HEALTH ALERT line in the journal, a status JSON at data/health-status.json for the dashboard to surface later, and a non-zero exit so the unit shows in `systemctl --failed`. OnBootSec=2min because a reboot is when this failure mode first appeared. Shared assertions live in scripts/lib/site-checks.sh so verify-deploy and health-check cannot drift apart. Two bugs were found and fixed in these checks by testing their failure paths, both of which would have made them report a broken site as healthy: * `$(curl ... || echo 000)` concatenates curl's own "000" transport-error output with the echoed one, giving "000000", which matched no case arm and was reported ok. * check_services sets FAILED_SERVICES, but health-check captured it with `$(...)`, so the array was populated in a subshell and discarded — silently disabling service recovery entirely. Both the calling convention and the subshell semantics are now pinned by tests. scripts/test-health-checks.sh: 9 cases, mostly negative, against a temp tree. Recovery additionally verified live on this host by stopping hawaii.service and by clearing scout's execute bit; both were detected, recovered and re-verified. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Stop deploy stripping the execute bit from site binaries

2026-09-20T00:25:17+00:00

The permission step ran find ${SITE_DIR} -type f -exec chmod 644 {} + chmod +x ${SITE_DIR}/app over the whole site directory, so it cleared +x on every other executable there and restored it only for doot's own app. Four binaries were caught: services/scout/app scout.service services/hawaii/app hawaii sub-service bin/claudomator claudomator@doot.terst.org.service bin/claude exec'd by claudomator via --claude-bin Nothing appears to break at deploy time: a running service keeps executing from its already-open file. The damage surfaces at the next reboot, when systemd cannot exec the binary and the unit dies with status=203/EXEC. scout.service went down that way at the 2026-08-25 reboot and stayed down three and a half weeks, serving 502s behind /scout/; claudomator has been crash-looping on the same error, 426k restarts and counting. Neither is alerted on, because doot itself is healthy and only those paths fail. Permission fixing is now scoped to the directories this script actually manages (public, templates, migrations) plus doot's own binary and .env. Every executable under services/*/app and bin/* is then re-asserted +x, since that is what an outage looks like when it is wrong and asserting it is free. Verified against a synthetic site tree: all five binaries keep 755, assets 644. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_018AwnALZL6bD4VBtATDVv3B

Fix production wedge: propagate context to Google Calendar API calls

2026-08-04T20:10:25+00:00

Three .Do() calls in google_calendar.go accepted a ctx parameter but never chained .Context(ctx) into the actual SDK call, so the existing global 60s request timeout never reached the blocking network call. One hung Google Calendar request wedged every DB/session-touching request path in production for three days (2026-08-01 through 2026-08-04), undetected because /health unconditionally returned 200 throughout. - Wire .Context(ctx) into GetUpcomingEvents, GetEventsByDateRange, and GetCalendarList. - Bound aggregateData's four external fetches with a per-fetch sub-context as defense-in-depth (only effective if the callee actually honors ctx -- documented as such, not oversold). - Make GetUpcomingEvents/GetEventsByDateRange fetch calendars concurrently instead of sequentially: a review of this fix caught that a shared per-fetch deadline over a sequential loop would starve calendars past the first under any real latency, silently caching partial results as complete. Concurrent fetches give every calendar an equal shot at the same deadline instead. - /health now does a real PingContext DB check instead of a static "ok" (Handler.PingDB, tested for both healthy and closed-DB cases). - Add internal/api/context_audit_test.go: an AST-based structural guard that fails any future .Do() call in google_*.go missing .Context(...) anywhere in its chain, so this class of bug can't silently recur. Verified by deliberately reintroducing the original bug against a backup and confirming the guard catches it. - Add scripts/health-watchdog.sh: cron job restarts the service if /health fails twice in a row, five minutes apart. go test ./... -race is green. Deployed and live-verified.

Guard deploy script to only run on tungsten

2026-07-17T22:24:08+00:00

Refuses to run if hostname doesn't match the documented deploy host, preventing an accidental deploy from a sandbox/dev environment. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01EZ7ikw2ukGJFTHE3bJS7zL

chore: unify and centralize agent configuration in .agent/

2026-03-22T23:45:19+00:00

fix: cd to repo root in deploy script

2026-03-22T09:41:33+00:00

Script uses relative paths (npm, go build, rsync) so it must run from the repo root regardless of the caller's working directory. Co-Authored-By: Claude Sonnet 4.6

feat: Phase 1 — remove bug feature and dead code

2026-03-21T21:24:02+00:00

- Delete Bug struct, BugToAtom, SourceBug, TypeBug, TypeNote - Remove bug store methods (SaveBug, GetBugs, ResolveBug, etc.) - Remove HandleGetBugs, HandleReportBug, bug branches in handlers - Remove bug routes, bugs.html template, bug UI from index.html - Remove AddMealToPlanner stub + interface method - Migration 018: DROP TABLE IF EXISTS bugs Co-Authored-By: Claude Sonnet 4.6

chore: migrate deploy + ops scripts from SSH to local execution

2026-03-16T08:53:44+00:00

Replace SSH-based deploy.sh with scripts/deploy that runs locally on the server. Update scripts/bugs, scripts/logs, and scripts/resolve-bug to drop SSH wrappers. Update CLAUDE.md and DESIGN.md to reflect new local workflow. Co-Authored-By: Claude Sonnet 4.6

Add build version footer, deploy ldflags, template test helper, and logs script

2026-02-08T01:42:07+00:00

- Display build commit hash in unobtrusive footer overlay - Inject buildCommit/buildTime via ldflags in deploy.sh - Add assertTemplateContains test helper, refactor existing template tests - Add scripts/logs for fetching production journalctl via SSH Co-Authored-By: Claude Opus 4.6