Skip to content
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
d083173
fix(onboard): preserve legacy gateway identity
HOYALIM Aug 12, 2026
40d9907
merge: refresh #8920 from main
prekshivyas Aug 13, 2026
68441ea
test(onboard): keep legacy gateway fixtures linear
prekshivyas Aug 13, 2026
3a8b35a
Merge branch 'main' into codex/fix-8740-legacy-gateway-identity
prekshivyas Aug 13, 2026
7bdab57
Merge branch 'main' into codex/fix-8740-legacy-gateway-identity
cv Aug 14, 2026
b2f10c1
fix(uninstall): revalidate scoped gateway cleanup
rsliter Aug 13, 2026
a7f60f7
Merge remote-tracking branch 'refs/remotes/live/main' into codex/pr-8…
rsliter Aug 14, 2026
b6b9ed0
docs(security): define recovery identity checks
rsliter Aug 14, 2026
2bc0c52
Merge remote-tracking branch 'refs/remotes/live/main' into codex/pr-8…
rsliter Aug 14, 2026
4971870
test(readiness): require fresh listener proof
rsliter Aug 14, 2026
4cf2cc9
fix(readiness): distinguish fresh listener checks
rsliter Aug 14, 2026
a1e3a96
test(uninstall): prove scoped dual-station gateway
cv Aug 14, 2026
305b3eb
fix(uninstall): bind external gateway process
cv Aug 14, 2026
c560c0d
fix(uninstall): restore namespace helper import
cv Aug 14, 2026
c83ebc9
test(uninstall): prove scoped gateway fixtures
cv Aug 14, 2026
f7b9e18
test(uninstall): prove remaining scoped fixtures
cv Aug 14, 2026
d331e09
test(uninstall): keep fixtures linear
cv Aug 14, 2026
c97f88f
test(uninstall): complete scoped gateway fixture coverage
rsliter Aug 14, 2026
4402a85
Merge remote-tracking branch 'origin/main' into codex/fix-8740-legacy…
cv Aug 15, 2026
17c9ed9
fix(onboard): clarify gateway config rejection
cv Aug 15, 2026
5337ade
docs(onboard): bound gateway backup retention
cv Aug 15, 2026
416726b
merge(main): incorporate current required checks
cv Aug 15, 2026
14eee41
merge(main): incorporate current required checks
cv Aug 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 33 additions & 7 deletions docs/manage-sandboxes/uninstall-nemoclaw.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ If OpenShell resource cleanup fails, uninstall exits nonzero and preserves the m
If marked Linux unit cleanup fails, uninstall exits nonzero before it scans for or stops a remaining gateway process or continues with later Docker and gateway-state cleanup.
OpenShell resource and Linux unit cleanup can partially complete before either failure.
After selected sandbox cleanup succeeds, uninstall removes those entries from `sandboxes.json` before gateway registration and Linux unit cleanup.
If a later step fails, the retry skips gateway selection and resumes the remaining cleanup.
If a later step fails, the retry skips the completed sandbox deletions and resumes the remaining cleanup.
Resolve the reported error.
Inspect the remaining gateways with `openshell gateway list`.
Rerun `NEMOCLAW_GATEWAY_PORT=<port> $$nemoclaw uninstall` with the gateway port from the failed uninstall.
Expand All @@ -119,19 +119,45 @@ Uninstall does not stop an `openshell-gateway` process that another non-root use
It names the owner and process ID, leaves that process running, and continues with the remaining cleanup.
If no other cleanup fails, uninstall exits with status `0` even though that process can keep its port in use.
Uninstall still tries to stop a `root`-owned process and the gateway process that this installation recorded.
If either of those stops fails, uninstall prints `sudo kill -9 <pid>` for the process.
If either stop fails, uninstall reports the process without printing a reusable privileged kill command.
Do not signal a PID from saved output.
Immediately before a privileged stop, verify that the live process owner and `openshell-gateway` command line match the exact gateway name and port.
Also prove that the PID file, runtime marker, and loaded sandbox namespace still match the selected state directory.
Rerun uninstall after the process stops.
A gateway-scoped uninstall and every `--all-gateway-ports` pass exit nonzero after that failure.
A single full uninstall reports the process and continues.
Before scoped cleanup stops a Docker gateway process, including a managed default gateway service, NemoClaw requires two Docker namespace proofs.
The selected Docker gateway configuration and any running gateway process must use the state-root-specific OpenShell sandbox namespace that NemoClaw generated.
Before each sandbox deletion during scoped Docker cleanup, NemoClaw proves the selected configuration and running gateway identity again and passes the selected gateway name to OpenShell.
The configuration and running process must use the state-root-specific OpenShell sandbox namespace that NemoClaw generated.
For a NemoClaw-managed gateway, the live proof also binds the process owner, PID file, runtime marker, and command line to the exact gateway name and port.
For an externally supervised gateway, NemoClaw proves the configured state.
It binds the supervisor's current main process to its owner, loaded sandbox namespace, declared executable, selected gateway name, and selected port.
When NemoClaw can prove an owner-private, generated configuration and complete JWT bundle that predate state-root scoping, restart keeps the legacy gateway ID, JWT bundle, and Docker driver's `default` namespace.
That compatibility keeps the gateway able to find existing containers and keeps their non-expiring sandbox JWT issuer valid.
NemoClaw regenerates the other gateway settings from the current runtime configuration.
For a proven legacy Podman gateway, NemoClaw preserves the gateway ID that existing sandbox JWTs use; the supported Podman schema has no `sandbox_namespace` setting to preserve.
If the existing identity is ambiguous or unsafe, or durable gateway state remains without its configuration, restart fails closed without rewriting the configuration or JWT bundle.
Fresh state roots and already scoped configurations continue to use the state-root-specific identity.
The legacy `default` namespace is not isolated across gateways, so it cannot satisfy the scoped-uninstall proof while sibling gateways remain.
Scoped uninstall stops before it deletes a sandbox, registry row, or gateway registration and preserves the selected gateway's runtime evidence and local state.
Because the supported OpenShell Podman schema does not expose `sandbox_namespace`, scoped Podman uninstall fails closed before signaling and preserves the gateway runtime evidence and local state.
Full single-gateway Podman uninstall continues to use normal graceful teardown.
For Docker, if either proof is absent, uninstall exits nonzero before it signals the host gateway.
For Docker, if any proof is absent, uninstall exits nonzero before it signals the host gateway.
NemoClaw preserves the gateway runtime evidence and local state.
Keep that state intact.
Restore the selected Docker gateway through the supported install or onboarding recovery flow so it restarts with the generated configuration.
For an already scoped gateway with stale runtime evidence, restore it through the supported install or onboarding recovery flow, verify the generated identity, and retry.
A proven legacy gateway is not silently converted by onboarding.
To retire one, first remove sibling gateways through their own proven scoped cleanup, verify that only the legacy gateway remains, and then use the full single-gateway uninstall path.
For an ambiguous or incomplete identity, stop the gateway and restore the exact generated `openshell-gateway.toml` and complete `jwt/` directory from a dedicated host-level backup path, represented here as `<gateway-identity-backup>`.
The backup must have been captured from that gateway's exact state directory before the failure and kept under the owning user's exclusive access.
Keep the `<gateway-identity-backup>` directory and its nested `jwt/` directory at mode `0700`, and keep the configuration and JWT files at mode `0600`.
The default gateway stores them under `~/.local/state/nemoclaw/openshell-docker-gateway/`; a non-default gateway uses `~/.local/state/nemoclaw/openshell-docker-gateway-<port>/`.
Restore them as the owning user.
Keep the gateway state root and its `jwt/` directory at mode `0700`, and do not grant group or other access to the configuration or JWT files.
NemoClaw does not reconstruct gateway identity from sandbox snapshots or `backup-all`; if no matching gateway-state backup exists, keep the state intact rather than attempting a scoped cleanup.
Verify every gateway with `openshell gateway list`.
Retry the scoped uninstall.
Retain `<gateway-identity-backup>` only until that command reports the restored gateway and the affected existing sandboxes authenticate successfully.
Then remove that dedicated backup directory as the owning user and verify its absence by replacing the placeholder in `test ! -e '<gateway-identity-backup>'` with the full backup path.
If verification fails, keep the backup under the same access restrictions and stop.
Do not add `sandbox_namespace` manually to a live gateway configuration because the running process can still be using its previous namespace.

<Note>
Expand Down
40 changes: 33 additions & 7 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4328,7 +4328,7 @@ If OpenShell resource cleanup fails, uninstall exits nonzero and preserves the m
If marked Linux unit cleanup fails, uninstall exits nonzero before it scans for or stops a remaining gateway process or continues with later Docker and gateway-state cleanup.
OpenShell resource and Linux unit cleanup can partially complete before either failure.
After selected sandbox cleanup succeeds, uninstall removes those entries from `sandboxes.json` before gateway registration and Linux unit cleanup.
If a later step fails, the retry skips gateway selection and resumes the remaining cleanup.
If a later step fails, the retry skips the completed sandbox deletions and resumes the remaining cleanup.
Resolve the reported error.
Inspect the remaining gateways with `openshell gateway list`.
Rerun `NEMOCLAW_GATEWAY_PORT=<port> $$nemoclaw uninstall` with the gateway port from the failed uninstall.
Expand All @@ -4339,19 +4339,45 @@ Uninstall does not stop an `openshell-gateway` process that another non-root use
It names the owner and process ID, leaves that process running, and continues with the remaining cleanup.
If no other cleanup fails, uninstall exits with status `0` even though that process can keep its port in use.
Uninstall still tries to stop a `root`-owned process and the gateway process that this installation recorded.
If either of those stops fails, uninstall prints `sudo kill -9 <pid>` for the process.
If either stop fails, uninstall reports the process without printing a reusable privileged kill command.
Do not signal a PID from saved output.
Immediately before a privileged stop, verify that the live process owner and `openshell-gateway` command line match the exact gateway name and port.
Also prove that the PID file, runtime marker, and loaded sandbox namespace still match the selected state directory.
Rerun uninstall after the process stops.
A gateway-scoped uninstall and every `--all-gateway-ports` pass exit nonzero after that failure.
A single full uninstall reports the process and continues.
Before scoped cleanup stops a Docker gateway process, including a managed default gateway service, NemoClaw requires two Docker namespace proofs.
The selected Docker gateway configuration and any running gateway process must use the state-root-specific OpenShell sandbox namespace that NemoClaw generated.
Before each sandbox deletion during scoped Docker cleanup, NemoClaw proves the selected configuration and running gateway identity again and passes the selected gateway name to OpenShell.
The configuration and running process must use the state-root-specific OpenShell sandbox namespace that NemoClaw generated.
For a NemoClaw-managed gateway, the live proof also binds the process owner, PID file, runtime marker, and command line to the exact gateway name and port.
For an externally supervised gateway, NemoClaw proves the configured state.
It binds the supervisor's current main process to its owner, loaded sandbox namespace, declared executable, selected gateway name, and selected port.
When NemoClaw can prove an owner-private, generated configuration and complete JWT bundle that predate state-root scoping, restart keeps the legacy gateway ID, JWT bundle, and Docker driver's `default` namespace.
That compatibility keeps the gateway able to find existing containers and keeps their non-expiring sandbox JWT issuer valid.
NemoClaw regenerates the other gateway settings from the current runtime configuration.
For a proven legacy Podman gateway, NemoClaw preserves the gateway ID that existing sandbox JWTs use; the supported Podman schema has no `sandbox_namespace` setting to preserve.
If the existing identity is ambiguous or unsafe, or durable gateway state remains without its configuration, restart fails closed without rewriting the configuration or JWT bundle.
Fresh state roots and already scoped configurations continue to use the state-root-specific identity.
The legacy `default` namespace is not isolated across gateways, so it cannot satisfy the scoped-uninstall proof while sibling gateways remain.
Scoped uninstall stops before it deletes a sandbox, registry row, or gateway registration and preserves the selected gateway's runtime evidence and local state.
Because the supported OpenShell Podman schema does not expose `sandbox_namespace`, scoped Podman uninstall fails closed before signaling and preserves the gateway runtime evidence and local state.
Full single-gateway Podman uninstall continues to use normal graceful teardown.
For Docker, if either proof is absent, uninstall exits nonzero before it signals the host gateway.
For Docker, if any proof is absent, uninstall exits nonzero before it signals the host gateway.
NemoClaw preserves the gateway runtime evidence and local state.
Keep that state intact.
Restore the selected Docker gateway through the supported install or onboarding recovery flow so it restarts with the generated configuration.
For an already scoped gateway with stale runtime evidence, restore it through the supported install or onboarding recovery flow, verify the generated identity, and retry.
A proven legacy gateway is not silently converted by onboarding.
To retire one, first remove sibling gateways through their own proven scoped cleanup, verify that only the legacy gateway remains, and then use the full single-gateway uninstall path.
For an ambiguous or incomplete identity, stop the gateway and restore the exact generated `openshell-gateway.toml` and complete `jwt/` directory from a dedicated host-level backup path, represented here as `<gateway-identity-backup>`.
The backup must have been captured from that gateway's exact state directory before the failure and kept under the owning user's exclusive access.
Keep the `<gateway-identity-backup>` directory and its nested `jwt/` directory at mode `0700`, and keep the configuration and JWT files at mode `0600`.
The default gateway stores them under `~/.local/state/nemoclaw/openshell-docker-gateway/`; a non-default gateway uses `~/.local/state/nemoclaw/openshell-docker-gateway-<port>/`.
Restore them as the owning user.
Keep the gateway state root and its `jwt/` directory at mode `0700`, and do not grant group or other access to the configuration or JWT files.
NemoClaw does not reconstruct gateway identity from sandbox snapshots or `backup-all`; if no matching gateway-state backup exists, keep the state intact rather than attempting a scoped cleanup.
Verify every gateway with `openshell gateway list`.
Retry the scoped uninstall.
Retain `<gateway-identity-backup>` only until that command reports the restored gateway and the affected existing sandboxes authenticate successfully.
Then remove that dedicated backup directory as the owning user and verify its absence by replacing the placeholder in `test ! -e '<gateway-identity-backup>'` with the full backup path.
If verification fails, keep the backup under the same access restrictions and stop.
Do not add `sandbox_namespace` manually to a live gateway configuration because the running process can still be using its previous namespace.

##### Uninstalling Every Gateway Port
Expand Down
29 changes: 17 additions & 12 deletions docs/reference/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -320,20 +320,21 @@ The sweep only targets processes owned by the current user whose command line ma

If onboarding preflight resolves the complete listener set for a gateway port conflict, the diagnostic lists every listener.
Each entry contains the process name and PID, or only the PID when NemoClaw cannot read the process name.
The stop command targets only listeners that fail ownership verification.
It does not target a verified managed listener.
The diagnostic identifies listeners that fail ownership verification, but it does not print a reusable process-stop command.
If NemoClaw resolves no listener, the diagnostic provides an `lsof` inspection command.
Before you run a stop command, confirm that each targeted listener is not part of a second NemoClaw gateway environment.
Before you stop a listener, confirm that it is not part of a second NemoClaw gateway environment.
Release that environment with `NEMOCLAW_GATEWAY_PORT=<port> $$nemoclaw uninstall` instead of stopping its process.

If a non-NemoClaw process is already bound to the dashboard port or the gateway port, identify the conflicting process, verify it is safe to stop, and terminate it:
If a non-NemoClaw process is already bound to the dashboard port or the gateway port, identify the conflicting process.
Stop it only when its command line names the application you intend to stop, you own the process or administer its service, and the application has no active work:

```bash
sudo lsof -i :18789
kill <PID>
sudo lsof -i :18789 -sTCP:LISTEN -P -n
```

If the process does not exit, use `kill -9 <PID>` to force-terminate it.
Stop it through its service manager when one owns it.
Otherwise, repeat the listener check immediately before you signal only the PID from that fresh result.
Repeat the check again before any `SIGKILL`.
Then retry onboarding.

Alternatively, override the conflicting port instead of stopping the other process.
Expand Down Expand Up @@ -1077,13 +1078,16 @@ Re-running onboard then fails preflight with `Port 18789 is not available.
Blocked by: ssh`.

Current NemoClaw detects this case and kills the orphaned SSH process automatically before retrying the port check.
If you see the error on an older release, identify the SSH process and terminate it manually:
If you see the error on an older release, identify the SSH process.
Use fresh listener output to confirm that your user owns the process and that it still listens on local port `18789`:

```bash
sudo lsof -i :18789
kill <PID>
sudo lsof -i :18789 -sTCP:LISTEN -P -n
```

Inspect the process separately with `ps -p <PID> -o user=,args=`.
Stop it only when the owner is your user, the command line is the stale SSH port forward for local port `18789`, and no active terminal or file-transfer session uses that process.
Repeat the listener check immediately before you signal only the PID from that fresh result.
Then re-run `$$nemoclaw onboard`.

</AgentOnly>
Expand Down Expand Up @@ -2976,11 +2980,12 @@ Run `fix-coredns.sh` to point CoreDNS at the container gateway IP instead, then
### `k3s` cannot find a freshly built image

After building a new sandbox image, `k3s` inside the gateway container sometimes fails to pull it even though the image exists on the host.
Remove the gateway registration, stop any leftover host gateway process, then re-run setup.
Remove the gateway registration, then resume onboarding.
If a privileged host gateway remains, do not use a host-wide process match.
Verify its live owner, exact gateway name and port, command line, PID file, runtime marker, and loaded sandbox namespace immediately before you stop it.

```bash
openshell gateway remove nemoclaw
sudo pkill -f openshell-gateway
$$nemoclaw onboard --resume
```

Expand Down
Loading
Loading