fix(db): keep the PostgreSQL volume name stable across major upgrades

.changeset/stable-postgres-volume-name.md

@@ -0,0 +1,5 @@
+---
+"hephaestus": minor
+---
+
+**Operators:** the PostgreSQL Compose volume keeps its stable `postgresql-data` name across the 17 → 18 upgrade instead of moving to a version-suffixed `postgresql-data-v18`. This corrects the PostgreSQL 18 qualification shipping in this same release, so no deployed instance ever sees the `-v18` name. The upgrade is dump, remove the volume, restore into the freshly initialized PostgreSQL 18 cluster; a PostgreSQL 18 container started against un-migrated PostgreSQL 17 data refuses to start instead of coming up empty.

.migration/stable-postgres-volume-name.md

@@ -0,0 +1,18 @@
+#### 🔴 The PostgreSQL upgrade keeps the stable volume name — rely on the verified dump, not a retained volume
+
+**Affected**: operators performing the PostgreSQL 17 → 18 migration this release requires. This
+entry corrects the "Restore PostgreSQL 17 data into PostgreSQL 18" entry above, which predates it.
+
+**Before**: this release was drafted to start PostgreSQL 18 on a new `postgresql-data-v18` volume,
+leaving the PostgreSQL 17 volume in place as the rollback path.
+
+**After**: the Compose volume keeps its stable `postgresql-data` name. The documented upgrade
+verifies the dump, removes the PostgreSQL 17 volume, and lets PostgreSQL 18 initialize a fresh
+cluster under the same name. Starting the new release without migrating is safe: a PostgreSQL 18
+container attached to PostgreSQL 17 data refuses to start rather than coming up healthy and empty.
+
+**Migration**: follow the current
+[Backup & Restore](https://docs.hephaestus.build/admin/backup-restore#postgresql-17-to-18)
+procedure. Where the entry above says to keep the PostgreSQL 17 volume until acceptance checks
+pass, keep the verified dump (with an off-host copy) instead — the old volume is removed during
+the upgrade, so the dump is the rollback artifact.

docker/compose.app.yaml

@@ -449,7 +449,7 @@ services:
           - postgres
     restart: unless-stopped
     volumes:
-      - postgresql-data-v18:/var/lib/postgresql
+      - postgresql-data:/var/lib/postgresql
     healthcheck:
       test:
         - CMD-SHELL
@@ -472,5 +472,5 @@ networks:
     driver: bridge
 
 volumes:
-  postgresql-data-v18:
+  postgresql-data:
   git-repos:

docker/preview/compose.app.yaml

@@ -22,7 +22,7 @@ services:
       POSTGRES_USER: hephaestus
       POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
     volumes:
-      - postgres-data-v18:/var/lib/postgresql
+      - postgres-data:/var/lib/postgresql
     security_opt:
       - no-new-privileges:true
     cap_drop:
@@ -376,4 +376,4 @@ networks:
     name: shared-network
 
 volumes:
-  postgres-data-v18:
+  postgres-data:

docs/admin/backup-restore.mdx

@@ -11,7 +11,7 @@ holds bounded evidence copies that may become irreproducible after upstream cont
 
 | Data | Where | Priority |
 | --- | --- | --- |
-| PostgreSQL volume | Compose-managed volume mounted by `postgres` | **Everything.** Accounts, workspaces, observations, feedback, encrypted provider tokens, JWT signing keys. |
+| PostgreSQL database | Compose-managed `postgresql-data` volume mounted by `postgres` | **Everything.** Accounts, workspaces, observations, feedback, encrypted provider tokens, JWT signing keys. |
 | `.env` | `/opt/hephaestus/docker/self-host/.env` | **Equal priority.** Its encryption keys protect values inside the database backup. Without them, provider credentials and signing material are unreadable. |
 | TLS certificates | `./letsencrypt/` | Optional — Let's Encrypt re-issues on first boot (rate limits permitting). |
 | Context Fabric | `git-repos` volume | Back up when replayable evidence is required. Skipping it discards repository clones and bounded replay/CAS content; durable job provenance remains in PostgreSQL, but the exact source bytes may be unrecoverable. |
@@ -57,13 +57,22 @@ timestamp.
 
 ## PostgreSQL 17 to 18
 
-PostgreSQL major-version data directories are not binary compatible. The PostgreSQL 18 image stores
-`PGDATA` under `/var/lib/postgresql/18/docker`.
-Do **not** point the PostgreSQL 18 container at the old PostgreSQL 17 volume and do not delete the
-old volume until the restored instance passes verification.
+PostgreSQL major-version data directories are not binary compatible, and the PostgreSQL 18 image
+stores `PGDATA` under `/var/lib/postgresql/18/docker`. The Compose volume keeps its stable
+`postgresql-data` name across the upgrade, so the move is: dump, destroy the volume, let
+PostgreSQL 18 initialize a fresh empty cluster under the same name, restore.
 
-From the PostgreSQL 17 release, stop application writes, make a logical dump, and record the exact
-Compose volume name:
+Skipping the procedure fails loudly, not silently: a PostgreSQL 18 container attached to the volume
+still holding PostgreSQL 17 data refuses to start (the entrypoint reports the old data and exits),
+so nothing is lost by starting the new release too early — the container just stays down until the
+steps below are completed.
+
+**Step (c) destroys the PostgreSQL 17 data.** From that point on, the dump is the only copy. Do not
+remove the volume before the dump has passed `pg_restore --list`, and copy the dump (and its
+checksum) off the host first.
+
+**(a)** From the running PostgreSQL 17 release, stop application writes and take a verified
+custom-format dump:
 
 ```bash
 cd /opt/hephaestus/docker/self-host
@@ -73,44 +82,72 @@ dc exec -T postgres pg_dump -U root -Fc hephaestus > /var/tmp/hephaestus-pg17.du
 test -s /var/tmp/hephaestus-pg17.dump
 dc exec -T postgres pg_restore --list < /var/tmp/hephaestus-pg17.dump >/dev/null
 sha256sum /var/tmp/hephaestus-pg17.dump > /var/tmp/hephaestus-pg17.dump.sha256
-PG17_VOLUME=$(docker inspect "$(dc ps -q postgres)" \
-  --format '{{range .Mounts}}{{if eq .Destination "/var/lib/postgresql/data"}}{{.Name}}{{end}}{{end}}')
-test -n "$PG17_VOLUME"
-dc down
-printf 'Preserved PostgreSQL 17 volume: %s\n' "$PG17_VOLUME"
 ```
 
-Check out the new release and prepare its signed release lock as described in [Upgrades](./install#upgrades).
-The supported self-host stack creates no custom roles or tablespaces; operators who added either must
-migrate those cluster-global objects separately. Restore into the new PostgreSQL 18 volume:
+Copy `/var/tmp/hephaestus-pg17.dump` and its `.sha256` off the host now.
+
+**(b)** Stop and remove the database container:
+
+```bash
+dc down postgres
+```
+
+**(c)** Remove the PostgreSQL 17 volume. Docker prefixes the Compose project name, so the exact
+name is `<project>_postgresql-data` — list the volumes and use the one ending in
+`_postgresql-data`:
+
+```bash
+docker volume ls --format '{{.Name}}' | grep '_postgresql-data$'
+docker volume rm <project>_postgresql-data
+```
+
+This deletes the PostgreSQL 17 cluster permanently.
+
+**(d)** Check out the new release and prepare its signed release lock as described in
+[Upgrades](./install#upgrades), then start the database alone. PostgreSQL 18 initializes a fresh,
+empty cluster in the recreated volume:
 
 ```bash
 dc() { docker compose --env-file .env --env-file release-lock.env "$@"; }
 dc up -d postgres
 until dc exec -T postgres pg_isready -U root -d hephaestus; do sleep 2; done
 test "$(dc exec -T postgres psql -U root -d hephaestus -Atc \
   "SELECT current_setting('server_version_num')::int / 10000 || ':' || (to_regclass('public.databasechangelog') IS NULL)")" = "18:true"
+```
+
+**(e)** Restore the dump. The supported self-host stack creates no custom roles or tablespaces;
+operators who added either must migrate those cluster-global objects separately:
+
+```bash
 sha256sum -c /var/tmp/hephaestus-pg17.dump.sha256
 dc exec -T postgres dropdb -U root hephaestus
 dc exec -T postgres createdb -U root hephaestus
 dc exec -T postgres pg_restore -U root -d hephaestus --no-owner --no-acl \
   --single-transaction < /var/tmp/hephaestus-pg17.dump
+```
+
+**(f)** Update the pg_partman extension to the version bundled with the new image, then start the
+stack:
+
+```bash
 dc exec -T postgres psql -U root -d hephaestus -v ON_ERROR_STOP=1 \
   -c 'ALTER EXTENSION pg_partman UPDATE'
 dc up -d --wait
 ```
 
-Verify application health, sign in, and inspect recent workspace activity. Then verify the database
-major, extension, and partition policy:
+**(g)** Verify application health, sign in, and inspect recent workspace activity. Then verify the
+database major, extension, partition policy, and that row counts in the tables you know best match
+the pre-upgrade state:
 
 ```bash
 dc exec -T postgres psql -U root -d hephaestus -Atc \
   "SELECT current_setting('server_version_num')::int / 10000 = 18; SELECT extversion = '5.5.0' FROM pg_extension WHERE extname = 'pg_partman'; SELECT count(*) = 1 FROM partman.part_config WHERE parent_table = 'public.auth_event';"
 ```
 
-Keep the dump and PostgreSQL 17 volume until acceptance checks pass and backup policy permits their
-deletion. Rollback means stopping PostgreSQL 18 and starting the prior release against the preserved
-volume; never run the prior application against the PostgreSQL 18 database.
+Keep the dump until acceptance checks pass and backup policy permits its deletion. Rollback means
+checking out the prior release, removing the volume again, starting its PostgreSQL 17 container
+against the recreated empty volume, and restoring the same dump; never run the prior application
+against the PostgreSQL 18 database.
 
 ### After a point-in-time restore
 

docs/decisions/0038-postgresql-18-release-baseline.md

@@ -2,7 +2,7 @@
 
 ## Status
 
-Accepted
+Accepted (amended 2026-08-31 — see the update below)
 
 ## Context
 
@@ -33,3 +33,27 @@ PostgreSQL 18. Preview environments create a fresh PostgreSQL 18 cluster.
 - [PostgreSQL major-upgrade documentation](https://www.postgresql.org/docs/18/upgrading.html)
 - [Official image `PGDATA` contract](https://github.com/docker-library/docs/blob/master/postgres/README.md#pgdata)
 - [pg_partman 5.5 release notes](https://github.com/pgpartman/pg_partman/releases/tag/v5.5.0)
+
+## Update — 2026-08-31 (PR #1673)
+
+One point of the decision is reversed before it ships in a release: the self-hosted Compose volume
+keeps its stable `postgresql-data` name (`postgres-data` in the preview stack) instead of moving to
+a version-suffixed `postgresql-data-v18`. Everything else above stands — PostgreSQL 18 as the
+qualified baseline, the `/var/lib/postgresql` mount, and the dump-and-restore upgrade.
+
+What the rename bought was a retained PostgreSQL 17 cluster for rollback. What it cost:
+
+- The residual risk flagged at review — with a renamed volume, an operator who upgrades without
+  migrating gets a stack that comes up **healthy but empty**, because Compose silently creates the
+  new volume. With the stable name the same mistake fails loudly: the 18+ image's entrypoint
+  detects the PostgreSQL 17 data in the volume and refuses to start (major-version data directories
+  are incompatible with the server), so nothing is silently lost and the operator is pointed back
+  at the documented migration.
+- Stable volume names are the Compose norm; a version-suffixed name forces a rename — and a fresh
+  round of this reasoning — at every future major.
+
+Rollback safety now rests on the verified dump rather than a retained volume:
+`docs/admin/backup-restore.mdx` § "PostgreSQL 17 to 18" requires `pg_restore --list` and an
+off-host copy before the volume is removed. `scripts/postgres-major-upgrade-test.ts` rehearses the
+stable-name flow (dump → destroy volume → fresh PostgreSQL 18 initialization → restore) and proves
+the loud-refusal property.

docs/decisions/README.md

@@ -76,7 +76,7 @@ do, its home is the Admin Guide (`docs/admin/`) and the runbook links to it.
 | [0035](0035-pull-request-previews-are-label-gated.md) | Pull request previews are label-gated and driven from the default branch | Accepted |
 | [0036](0036-agent-runtime-runs-on-node-24.md) | Agent runtime runs on Node.js 24 with bounded resources | Accepted |
 | [0037](0037-node-24-and-pnpm-12-are-the-javascript-toolchain.md) | Node.js 24 and pnpm 12 are the JavaScript toolchain | Accepted |
-| [0038](0038-postgresql-18-release-baseline.md) | PostgreSQL 18 is the qualified release baseline | Accepted |
+| [0038](0038-postgresql-18-release-baseline.md) | PostgreSQL 18 is the qualified release baseline | Accepted (amended 2026-08-31 #1673 — the Compose volume keeps its stable `postgresql-data` name) |
 | [0039](0039-git-and-postgresql-own-evidence.md) | Git owns repository evidence; PostgreSQL owns captured payloads and references | Accepted |
 
 Template: [0000-template.md](0000-template.md).

scripts/postgres-major-upgrade-test.ts

@@ -1,11 +1,17 @@
+// Rehearses the operator upgrade documented in docs/admin/backup-restore.mdx § PostgreSQL 17 to 18:
+// dump the PostgreSQL 17 cluster, destroy the volume, recreate it under the same stable name, and
+// restore into a fresh PostgreSQL 18 cluster. It also proves the safety property ADR 0038's
+// amendment leans on — PostgreSQL 18 refuses to start against PostgreSQL 17 data instead of coming
+// up healthy and empty.
 import { spawnSync } from "node:child_process";
 import { randomUUID } from "node:crypto";
 
 const id = `postgres-upgrade-${randomUUID().slice(0, 8)}`;
 const source = `${id}-17`;
 const target = `${id}-18`;
-const sourceVolume = `${source}-data`;
-const targetVolume = `${target}-data`;
+// One volume, reused across the major upgrade — the same stable `postgresql-data` name the shipped
+// compose files declare (Compose prefixes it with the project name).
+const volume = `${id}_postgresql-data`;
 
 function run(command: string, args: string[], input?: Buffer): string {
 	const result = spawnSync(command, args, { encoding: "utf8", input, maxBuffer: 64 * 1024 * 1024 });
@@ -54,7 +60,7 @@ function wait(container: string): void {
 	throw new Error(`${container} did not become ready`);
 }
 
-function start(container: string, volume: string, mount: string, image: string): number {
+function start(container: string, dataVolume: string, mount: string, image: string): number {
 	docker(
 		"run",
 		"-d",
@@ -63,7 +69,7 @@ function start(container: string, volume: string, mount: string, image: string):
 		"-p",
 		"127.0.0.1::5432",
 		"-v",
-		`${volume}:${mount}`,
+		`${dataVolume}:${mount}`,
 		"-e",
 		"POSTGRES_DB=hephaestus",
 		"-e",
@@ -96,10 +102,9 @@ try {
 		"docker/postgres",
 	]);
 	run("docker", ["build", "-t", `${id}:18`, "docker/postgres"]);
-	docker("volume", "create", sourceVolume);
-	docker("volume", "create", targetVolume);
+	docker("volume", "create", volume);
 
-	const sourcePort = start(source, sourceVolume, "/var/lib/postgresql/data", `${id}:17`);
+	const sourcePort = start(source, volume, "/var/lib/postgresql/data", `${id}:17`);
 	if (sql(source, "SHOW server_version_num").slice(0, 2) !== "17")
 		throw new Error("source is not PostgreSQL 17");
 
@@ -145,9 +150,41 @@ try {
 		input: dump,
 	});
 	if (listing.status !== 0) throw new Error("source dump is unreadable");
-	docker("stop", source);
+	docker("rm", "-f", source);
 
-	start(target, targetVolume, "/var/lib/postgresql", `${id}:18`);
+	// The safety property behind keeping the volume name stable: an operator who upgrades without
+	// completing the dump-and-restore gets a container that refuses to start, not a silently empty
+	// database. The 18+ entrypoint detects the foreign PG_VERSION and exits before initdb.
+	const refusal = spawnSync(
+		"docker",
+		[
+			"run",
+			"--rm",
+			"-v",
+			`${volume}:/var/lib/postgresql`,
+			"-e",
+			"POSTGRES_DB=hephaestus",
+			"-e",
+			"POSTGRES_USER=root",
+			"-e",
+			"POSTGRES_PASSWORD=root",
+			`${id}:18`,
+		],
+		{ encoding: "utf8", timeout: 120_000, maxBuffer: 64 * 1024 * 1024 },
+	);
+	if (refusal.status === 0 || refusal.status === null) {
+		throw new Error("PostgreSQL 18 did not refuse the PostgreSQL 17 data");
+	}
+	if (!refusal.stderr.includes("PostgreSQL data")) {
+		throw new Error(`PostgreSQL 18 failed for an unexpected reason:\n${refusal.stderr}`);
+	}
+
+	// The operator's destructive step: the PostgreSQL 17 volume is removed and recreated under the
+	// same name, so from here on the verified dump is the only copy of the data.
+	docker("volume", "rm", volume);
+	docker("volume", "create", volume);
+
+	start(target, volume, "/var/lib/postgresql", `${id}:18`);
 	docker("exec", target, "dropdb", "-U", "root", "hephaestus");
 	docker("exec", target, "createdb", "-U", "root", "hephaestus");
 	const restore = spawnSync(
@@ -193,15 +230,8 @@ try {
 		) !== "t"
 	)
 		throw new Error("auth_event partitions were not restored");
-
-	docker("stop", target);
-	docker("start", source);
-	wait(source);
-	if (sql(source, "SELECT value FROM upgrade_qualification WHERE id=1") !== "preserved")
-		throw new Error("PostgreSQL 17 rollback volume is unreadable");
 } finally {
 	for (const container of [source, target]) spawnSync("docker", ["rm", "-f", container]);
-	for (const volume of [sourceVolume, targetVolume])
-		spawnSync("docker", ["volume", "rm", "-f", volume]);
+	spawnSync("docker", ["volume", "rm", "-f", volume]);
 	for (const image of [`${id}:17`, `${id}:18`]) spawnSync("docker", ["rmi", "-f", image]);
 }