chore(release): automate container probes + document post-tag re-cut

[lfnovo]

Retro improvements captured immediately after the v1.14.0 release, while the context is fresh (Phase 9 of the release process).

What

1. Container probes are now automated (scripts/release-test/release-image-test.sh)

A new probe scenario, run as part of make release-test ... all, covers two release checks that a Python test suite structurally can't — they depend on the shipped image's process supervision (supervisord, entrypoint), not on app code:

• OPEN_NOTEBOOK_WORKER_MAX_TASKS reaches the in-image worker (feat(worker): expose OPEN_NOTEBOOK_WORKER_MAX_TASKS for worker concurrency #1141). supervisord's command= doesn't run through a shell, so the value is only honored via the sh -c wrapper — the probe boots with the var set and reads the concurrency back from the worker's startup log.
• The worker survives startup with HTTP_PROXY set ([Install]: HTTP proxy breaks worker startup — SurrealDB websocket tunneled through proxy (HTTP 403) #1160), the internal SurrealDB websocket isn't tunneled (no 403), and a user's NO_PROXY value is preserved. Boots against a real SurrealDB with a dead proxy configured and asserts the worker reaches RUNNING.

Both were run by hand during the v1.14.0 release. Each probe is a self-contained standalone container with its own cleanup, so a failure can't leak state into the fresh/upgrade scenarios.

2. Two gotchas documented (.github/RELEASE_PROCESS.md)

• Never leave the version bump uncommitted on a branch — during v1.14.0 the bump was carried across a branch switch and swept into an unrelated fix PR by git add -A, shipping the version inside a fix(...) commit.
• A fix found after the tag exists but before publication requires a full re-cut (move the tag to the new commit and rebuild/re-push the version images), not just a tag nudge — otherwise publication promotes the un-fixed artifact to v1-latest.

The exact re-cut command sequence was also added to the release runbook (maintainer-local, gitignored).

Verification

$ bash scripts/release-test/release-image-test.sh probe lfnovo/open_notebook:1.14.0
  ✅ OPEN_NOTEBOOK_WORKER_MAX_TASKS honored by the in-image worker
  ✅ worker starts with HTTP_PROXY set (internal ws not tunneled)
  ✅ no proxy-rejection (403) in worker/API startup
  ✅ user NO_PROXY value preserved (not clobbered)
═══ RESULT: 4 passed, 0 failed

bash -n clean on the script.

[cubic-dev-ai]

2 issues found across 3 files

Confidence score: 4/5

• The riskiest gap is in .github/RELEASE_PROCESS.md: it points maintainers to a runbook.md section for “Re-cut after a post-tag fix” that doesn’t exist, so a real post-tag incident could stall or lead to inconsistent manual recovery steps—add or link the exact re-cut procedure before merging.
• In scripts/release-test/release-image-test.sh, fixed Docker resource names can collide across parallel or interrupted runs, causing false probe failures and possible reuse/cleanup of another run’s resources—generate a per-invocation suffix and ensure cleanup targets only that run’s resources before merging.

_{You’re at about 95% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.}

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="scripts/release-test/release-image-test.sh">

<violation number="1" location="scripts/release-test/release-image-test.sh:187">
P2: Parallel or interrupted release-test runs can collide on these fixed Docker resource names, making the probe fail before testing the image and leaving/reusing another run's resources. Generate a per-invocation suffix for network and container names, and clean it via a trap.</violation>
</file>

<file name=".github/RELEASE_PROCESS.md">

<violation number="1" location=".github/RELEASE_PROCESS.md:164">
P2: The re-cut gotcha references `runbook.md` → "Re-cut after a post-tag fix", but that section does not exist. The file at `.agents/skills/release/runbook.md` has no re-cut or post-tag content — a maintainer facing a post-tag blocker will hunt for a non-existent section under pressure.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: Parallel or interrupted release-test runs can collide on these fixed Docker resource names, making the probe fail before testing the image and leaving/reusing another run's resources. Generate a per-invocation suffix for network and container names, and clean it via a trap.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At scripts/release-test/release-image-test.sh, line 187:

<comment>Parallel or interrupted release-test runs can collide on these fixed Docker resource names, making the probe fail before testing the image and leaving/reusing another run's resources. Generate a per-invocation suffix for network and container names, and clean it via a trap.</comment>

<file context>
@@ -140,10 +147,85 @@ print('yes' if any(n.get('name')=='release-probe' for n in nbs) else 'no')" 2>/d
+  # ws://). The app injects the internal hosts into NO_PROXY at startup. Boot
+  # with a dead proxy set plus a user NO_PROXY value, against a real DB, and
+  # assert the worker reaches RUNNING and the user's NO_PROXY value survives.
+  local NET=onrelprobe-net
+  docker network create "$NET" >/dev/null 2>&1
+  docker run -d --rm --network "$NET" --name onrelprobe-surreal \
</file context>

[cubic-dev-ai]

P2: The re-cut gotcha references runbook.md → "Re-cut after a post-tag fix", but that section does not exist. The file at .agents/skills/release/runbook.md has no re-cut or post-tag content — a maintainer facing a post-tag blocker will hunt for a non-existent section under pressure.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At .github/RELEASE_PROCESS.md, line 164:

<comment>The re-cut gotcha references `runbook.md` → "Re-cut after a post-tag fix", but that section does not exist. The file at `.agents/skills/release/runbook.md` has no re-cut or post-tag content — a maintainer facing a post-tag blocker will hunt for a non-existent section under pressure.</comment>

<file context>
@@ -148,6 +148,20 @@ accepted improvements immediately — update this document, the scripts under
+  `v1-latest`) and a blocker is found in bucket C, the tag must move to the new
+  commit AND the version images must be rebuilt — a stale tag or stale registry
+  image will otherwise be what publication promotes to `v1-latest`. The exact
+  sequence is in `runbook.md` → "Re-cut after a post-tag fix" (v1.14.0 lesson).
 - **RC stack on non-default ports needs `API_URL`** or the browser talks to
   `host:5055` — on a dev machine that is the development API (data crossover).
</file context>