1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
|
---
name: project-doot-prod
description: "Production deployment facts for doot.terst.org — deploy target is tungsten ONLY (migrated from titanium 2026-07-17); verified architecture facts as of the first real deploy on 2026-07-17"
metadata:
node_type: memory
type: project
originSessionId: 31afbee3-b8e9-4163-a1bf-ea5392aef019
---
**Update (2026-07-17): production migrated from titanium to tungsten.** The deploy host is now **tungsten** — the user confirmed this explicitly ("we migrated to tungsten so that is now the deploy host"). This same day, the first real deploy since the migration ran successfully from a tungsten sandbox and corrected several stale facts below (previously copied from a box that only *resembled* prod — see the cautionary tale). Everything in this file is now verified against that real deploy, not inferred from resemblance.
**Verification: run `hostname`.** Reports `tungsten` on the deploy host. Always check before any deploy-shaped action; if it's not `tungsten`, do the build/test/commit/push work and stop there, or ask. `scripts/deploy` (see below) now enforces this itself and refuses to run anywhere else — but still verify independently before invoking it, since the classifier gating deploy-shaped Bash actions wants fresh, visible confirmation in the same turn, not a memory citation.
**Cautionary tale (pre-migration, still the reason for the rule):** In one session, "deploy = build + restart" was performed on a sandbox that wasn't the deploy host: built the Go binary in place, ran DB migrations against its `dashboard.db`, restarted the service — believing this was production. It wasn't; had to be unwound (stop/disable service, restore DB from backup, delete the built binary). Lesson: **confirm the current host is the deploy host before any deploy-shaped action — don't infer it from environment resemblance.**
---
**Real deploy mechanism, confirmed 2026-07-17:** run `scripts/deploy` from the repo root (e.g. `/workspace/doot`). It now has a `DEPLOY_HOST="tungsten"` guard at the top that hard-exits if `hostname` doesn't match — added after that session's classifier explicitly gated the raw deploy steps and asked for this kind of self-enforcement. The script: builds CSS (`npm run css:build`), builds the Go binary with `-ldflags` version stamping, stops the service, rsyncs `web/static`→public, `web/templates`, and `migrations` into the site dir, moves the binary in, fixes ownership/perms, restarts the service, and prints `systemctl status`.
**Two other "deploy" scripts existed and were deleted 2026-07-17 as abandoned scaffolding:** `deployment/deploy` (a near-duplicate of `scripts/deploy`) and `deployment/post-receive` (a git post-receive hook meant to be installed at `/site/doot.terst.org/app-code/hooks/post-receive` for a push-to-deploy workflow) — confirmed the hook was never actually installed (no `.git/hooks/post-receive` existed) and the `app-code` checkout it depended on was stale (some files untouched since Jan, well predating current work). That checkout has since been `rm -rf`'d. `deployment/apache.conf` and `deployment/task-dashboard.service` are still sitting in the repo — known stale/needs-a-new-home per the user, but no destination decided yet, so left in place.
**Verified real paths (previous versions of this memory had these wrong — they were guessed from a resembling sandbox, not checked):**
- Binary: `/site/doot.terst.org/app` (NOT `/workspace/doot/app` — that's just the source checkout the deploy script builds from before moving the binary over)
- DB: `/site/doot.terst.org/data/dashboard.db` (NOT `/workspace/doot/dashboard.db`, and NOT `/site/doot.terst.org/dashboard.db` either — that top-level path is a stale 0-byte decoy file that's easy to query by mistake; the real one is one level deeper, in `data/`. Confirmed via `DATABASE_PATH` in `/site/doot.terst.org/.env`.)
- Env: `/site/doot.terst.org/.env`
- Service: **`task-dashboard@doot.terst.org.service`** (a templated unit, `/etc/systemd/system/task-dashboard@.service`) — NOT `doot.service`. `systemctl restart task-dashboard@doot.terst.org.service` / `journalctl -u task-dashboard@doot.terst.org.service`.
- Static/templates/migrations synced into `/site/doot.terst.org/public`, `/site/doot.terst.org/templates`, `/site/doot.terst.org/migrations` respectively by the deploy script.
- DEFAULT_PASS in .env is a placeholder (user already exists in DB; EnsureDefaultUser only runs if users table is empty)
**Fresh-sandbox gotcha: `node_modules` can look installed but be broken.** `npm install` reported "up to date, audited 107 packages" while the actual `postcss` CLI binary was non-functional (`Cannot find module '.../node_modules/.bin/lib/args.js'`), because `node_modules` only had 98 stale entries and was missing `postcss-cli` entirely. Fix: `rm -rf node_modules && npm ci` (there's a `package-lock.json`) before trusting `npm run css:build` / `scripts/deploy`.
**Migration system tracks by full filename, not numeric prefix** (`internal/store/sqlite.go` `runMigrations`, table `schema_migrations`). See [[feedback-migration-renumbering]] for the incident this caused (a renamed already-applied migration crash-looped the service after this deploy) and the rule going forward (never rename/renumber an existing migration file to resolve a merge collision).
**Git config:** `git config --global --add safe.directory /site/git.terst.org/repos/doot.git` has already been granted and applied (global, persists across sessions on this box) — pushes to the `local` remote from a root shell won't hit the "dubious ownership" error again for this specific path. A similar error will still occur for other www-data-owned paths (e.g. it happened for the now-deleted `/site/doot.terst.org/app-code`) until the same exception is added for them too, and that always needs to be asked for explicitly first (never run `git config --global` unprompted).
**Reverse proxy: Apache (NOT Caddy) — this memory previously had it backwards.** Corrected 2026-08-04 after checking live: `caddy.service`/`caddy-api.service` are both `disabled` and not running; Apache2 is what's actually listening on :80/:443 (confirmed via `ss -tlnp` and `apache2ctl -S`). The `doot/deployment/apache.conf` mentioned above as "stale/unused" was WRONG — it's real config, just not the one checked into the repo (live vhosts are `/etc/apache2/sites-available/doot.terst.org.conf` and `doot.terst.org-le-ssl.conf`, proxying `/` to `http://127.0.0.1:38080/` — the app's actual port per `PORT=38080` in `.env`, not 8080 as previously assumed here). Apache's `ProxyPass` config is separate from any TLS cert renewal mechanism, so also don't assume Caddy handles ACME either — check `/etc/letsencrypt/` (certbot) vs Caddy's automatic HTTPS before trusting either claim. This was learned the hard way: don't trust old infra claims in memory without a live check when debugging anything proxy/reachability related. See [[feedback-verify-before-asserting-root-cause]] for the fuller incident writeup this correction came out of.
- stash.terst.org → localhost:8000 (unverified whether this one's still Caddy or also moved to Apache — recheck before relying on it)
**Playground web demo:**
- /workspace/playground/web/server.py (stdlib Python, port 9090)
- Service: /etc/systemd/system/playground-web.service
- Exposed via doot gateway at /playground/ (PLAYGROUND_URL=http://127.0.0.1:9090 in doot .env)
**Other services:**
- stash: /workspace/stash (FastAPI/uvicorn, port 8000) — separate postgres on 5432
- Claudomator: not currently running; would be on port 8484
**Git remotes:** `github` (github.com:thepeterstone/doot.git) works from any sandbox. `local` (/site/git.terst.org/repos/doot.git) is a local filesystem path that only exists on the deploy host itself (tungsten) — its absence on a given box is itself a signal that box is NOT the deploy host.
|
