fix(portable): separate Podman activation readiness

docs/reference/commands.mdx

@@ -874,6 +874,15 @@ After all portable TCP probe attempts fail, onboarding prints commands for the u
 
 Onboarding prints the same commands when the portable probe cannot reach the user-scoped Podman service.
 The printed rerun command keeps the portable experimental profile selected.
+Portable commands reconstruct the current user's rootless Podman socket authority from NemoClaw state before they use the Docker-compatible API.
+They do not select an endpoint from ambient Docker or Podman runtime variables or named connections.
+When the user-scoped socket-backed service needs activation, NemoClaw activates it and waits through a bounded startup period for a real Podman API response.
+Only a successful response permits the shorter steady-state API health deadline.
+Onboarding and portable sandbox lifecycle commands use this same readiness contract.
+Failures identify socket authority, service activation, startup API health, or steady-state API health without reporting credentials.
+NemoClaw does not fall back to Docker or report an absent or unreachable endpoint as healthy.
+A successful cold path uses the `cold` timing label and reports activation, API, and total time in milliseconds.
+A successful warm path uses the `warm` timing label and reports steady-state API and total time in milliseconds.
 To tune the existing-gateway HTTP health poll, use `NEMOCLAW_REUSE_HEALTH_POLL_COUNT` (default `6`) and `NEMOCLAW_REUSE_HEALTH_POLL_INTERVAL` (default `5` seconds).
 The poll count is clamped to a minimum of `1` so the health probe always runs at least once, and the interval is clamped to a minimum of `0` (no sleep between attempts).
 
@@ -4958,6 +4967,7 @@ Defaults are sized for typical hardware; override only if you see false-positive
 | Variable | Default | Effect |
 |----------|---------|--------|
 | `NEMOCLAW_MCP_PROVIDER_SYNC_TIMEOUT_SECONDS` | `30` | Maximum time to wait for an OpenShell MCP provider credential revision to become active or fully revoked inside the sandbox. Integer seconds; raise only when provider synchronization is unusually slow. |
+| `NEMOCLAW_PORTABLE_PODMAN_STARTUP_TIMEOUT_MS` | `60000` | Maximum cold-start time for portable rootless Podman socket activation and the first real API response. Set an integer from `15000` through `300000` milliseconds. This setting does not change the fixed 10,000 ms steady-state API deadline. |
 | `NEMOCLAW_SANDBOX_EXEC_TIMEOUT_MS` | per call site (typically `15000`) | Overrides the default timeout for `openshell sandbox exec` calls issued by recovery and lifecycle helpers. Integer milliseconds; non-positive or non-numeric values fall back to the per-call-site default. |
 | `NEMOCLAW_STATUS_PROBE_TIMEOUT_MS` | built-in default | Overrides the timeout for the OpenShell status probe used by `$$nemoclaw <name> status`. Integer milliseconds; non-positive or non-numeric values fall back to the default. |
 | `NEMOCLAW_WSL_GPU_PROOF_TIMEOUT_MS` | `180000` | Maximum time for the bounded Docker CUDA workload on an eligible ARM64 Linux host. A positive finite number of milliseconds overrides the default. Invalid, infinite, zero, and negative values use the default. |

docs/reference/troubleshooting.mdx

@@ -3329,6 +3329,60 @@ Then rerun portable onboarding:
 $$nemoclaw onboard --experimental-profile portable
 ```
 
+### Portable Podman Readiness Fails
+
+Portable commands use the current user's rootless Podman socket authority recorded in NemoClaw state.
+They ignore ambient Docker and Podman runtime selectors, including named connections.
+Do not export another `DOCKER_HOST`, `DOCKER_CONTEXT`, `CONTAINER_HOST`, or `CONTAINER_CONNECTION` to bypass a readiness failure.
+
+A portable readiness failure identifies the stage that did not complete:
+
+| Stage | Meaning | Recovery |
+| --- | --- | --- |
+| Socket authority | The recorded socket is unsafe, is replaced, or does not match the current user's recorded runtime authority. | Stop and inspect the reported path. Restore the recorded current-user runtime instead of selecting another endpoint. |
+| Service activation | The current user's systemd manager could not activate the Podman socket-backed service, or activation did not create the recorded socket within the startup period. | Inspect `podman.socket` and `podman.service` with the commands below. Correct the reported user-unit failure, then rerun the NemoClaw command. |
+| Startup API health | The service was activated, but the recorded endpoint did not return a real Podman API response within the startup period. | Inspect the user-unit logs and run the explicit API request below against the reported socket path. Raise `NEMOCLAW_PORTABLE_PODMAN_STARTUP_TIMEOUT_MS` only when valid cold activation needs more than 60,000 ms. |
+| Steady-state API health | An endpoint that completed activation did not answer the later shorter health check. | Inspect host load and the user-unit logs, then rerun the NemoClaw command. |
+
+Inspect the current user's units without changing them:
+
+```bash
+systemctl --user status podman.socket podman.service --no-pager
+journalctl --user -u podman.socket -u podman.service --since -10m --no-pager
+```
+
+If the units need activation, restart the active service when present and enable the socket:
+
+```bash
+systemctl --user try-restart podman.service
+systemctl --user enable --now podman.socket
+```
+
+Use the exact socket path from the NemoClaw failure to require a real server response:
+
+```bash
+podman --remote \
+  --url unix://<recorded-socket-path> \
+  version \
+  --format 'Server Version: {{.Server.Version}}'
+```
+
+Continue only when the command exits with status `0` and prints a nonempty server version.
+The request and the readiness report contain no credentials.
+Rerun the original NemoClaw command without exporting a Docker or Podman runtime selector.
+
+If valid cold activation needs a larger budget, set an integer from `15000` through `300000` milliseconds:
+
+```bash
+export NEMOCLAW_PORTABLE_PODMAN_STARTUP_TIMEOUT_MS=120000
+$$nemoclaw <original-arguments>
+```
+
+The default cold-start budget is 60,000 ms.
+The later steady-state API deadline is fixed at 10,000 ms and does not use this setting.
+A successful cold path uses the `cold` timing label and reports activation, API, and total time in milliseconds.
+A successful warm path uses the `warm` timing label and reports steady-state API and total time in milliseconds.
+
 ### Portable Host Gateway Is Unreachable
 
 The portable experimental profile maps `host.openshell.internal` to the OpenShell Podman host gateway.
@@ -3368,6 +3422,9 @@ Expected output:
 active
 ```
 
+An active socket alone does not establish API health.
+Run the explicit Podman API request from [Portable Podman Readiness Fails](#portable-podman-readiness-fails) before you rerun onboarding.
+
 Then rerun portable onboarding:
 
 ```bash

src/lib/actions/sandbox/doctor-lifecycle-registration.ts

@@ -5,6 +5,7 @@ import {
   collectLifecycleRegistrationIssues,
   type LifecycleRegistrationIssue,
 } from "../../domain/lifecycle-registration";
+import { inspectPortableRuntimeReceiptReadiness } from "../../onboard/experimental/portable-runtime-receipt-readiness";
 import type { SandboxEntry } from "../../state/registry";
 import type { DoctorCheck } from "./doctor-report";
 
@@ -23,6 +24,28 @@ function formatFieldList(
     .join(", ");
 }
 
+export function buildPortableRuntimeCheck(sandboxName: string): DoctorCheck | null {
+  const portable = inspectPortableRuntimeReceiptReadiness(sandboxName);
+  if (!portable) return null;
+  const recordedSocket = !portable.ok && portable.socketPath
+    ? ` Recorded socket: ${portable.socketPath}.`
+    : "";
+  return portable.ok
+    ? {
+        group: "Host",
+        label: "Portable Podman API",
+        status: "ok",
+        detail: `server ${portable.serverVersion}; ${portable.timing.mode}; activation ${String(portable.timing.activationMs)} ms; API ${String(portable.timing.apiMs)} ms; total ${String(portable.timing.totalMs)} ms`,
+      }
+    : {
+        group: "Host",
+        label: "Portable Podman API",
+        status: "fail",
+        detail: `${portable.stage}: ${portable.detail}${recordedSocket}`,
+        hint: "repair the recorded current-user Podman endpoint, then retry",
+      };
+}
+
 export function buildLifecycleRegistrationCheck(
   sandboxName: string,
   entry: SandboxEntry,

src/lib/actions/sandbox/doctor.ts

@@ -49,7 +49,10 @@ import {
   type DoctorInferenceRoute,
   resolveDoctorReasoningEffort,
 } from "./doctor-inference";
-import { buildLifecycleRegistrationCheck } from "./doctor-lifecycle-registration";
+import {
+  buildLifecycleRegistrationCheck,
+  buildPortableRuntimeCheck,
+} from "./doctor-lifecycle-registration";
 import { collectMessagingDoctorChecks } from "./doctor-messaging";
 import {
   buildDoctorReport,
@@ -136,36 +139,44 @@ function cliBuildCheck(): DoctorCheck {
   };
 }
 
-function collectHostChecks(sb: SandboxEntry | null | undefined): {
-  checks: DoctorCheck[];
-  openshellBin: ReturnType<typeof resolveOpenshell>;
-} {
-  const cli = cliBuildCheck();
-  const openshellBin = resolveOpenshell();
-  let runtimeCheck: DoctorCheck;
+function inspectRuntimeHost(sb: SandboxEntry | null | undefined): DoctorCheck {
+  const portable = sb ? buildPortableRuntimeCheck(sb.name) : null;
+  if (portable) return portable;
+  const recorded = sb?.openshellDriver?.trim();
+  const provider = recorded
+    ? requireRuntimeProviderBundle(recorded, CURRENT_RUNTIME_PROVIDER_BUNDLES)
+    : resolveCurrentRuntimeProviderBundle();
+  return provider.preflightDoctor.inspectHost();
+}
+
+function runtimeHostCheck(sb: SandboxEntry | null | undefined): DoctorCheck {
   try {
-    const recorded = sb?.openshellDriver?.trim();
-    const provider = recorded
-      ? requireRuntimeProviderBundle(recorded, CURRENT_RUNTIME_PROVIDER_BUNDLES)
-      : resolveCurrentRuntimeProviderBundle();
-    runtimeCheck = provider.preflightDoctor.inspectHost();
+    return inspectRuntimeHost(sb);
   } catch (error) {
     const detail =
       error instanceof RuntimeProviderSelectionError
         ? error.message
         : `Runtime provider inspection failed: ${error instanceof Error ? error.message : String(error)}`;
-    runtimeCheck = {
+    return {
       group: "Host",
       label: "Runtime provider",
       status: "fail",
       detail,
       hint: "restore a supported durable runtime provider identity before retrying",
     };
   }
+}
+
+function collectHostChecks(sb: SandboxEntry | null | undefined): {
+  checks: DoctorCheck[];
+  openshellBin: ReturnType<typeof resolveOpenshell>;
+} {
+  const cli = cliBuildCheck();
+  const openshellBin = resolveOpenshell();
   return {
     checks: [
       cli,
-      runtimeCheck,
+      runtimeHostCheck(sb),
       {
         group: "Host",
         label: "OpenShell CLI",

src/lib/actions/sandbox/gateway-failure-classifier.ts

@@ -9,11 +9,20 @@ import { CLI_NAME } from "../../cli/branding";
 import { GATEWAY_PORT } from "../../core/ports";
 import { resolveSandboxContainerOwner } from "../../domain/sandbox/container-owner";
 import { resolveGatewayPortFromName } from "../../onboard/gateway-binding";
+import type { PortablePodmanReadinessResult } from "../../onboard/experimental/portable-runtime-readiness";
+import {
+  inspectPortableRuntimeReceiptReadiness,
+  type PortableRuntimeReceiptReadinessDeps,
+} from "../../onboard/experimental/portable-runtime-receipt-readiness";
 import * as registry from "../../state/registry";
 import { getSandboxTargetGatewayName } from "./gateway-target";
 
 const DOCKER_TIMEOUT_MS = 3000;
 const PORT_PROBE_TIMEOUT_MS = 2000;
+const portableRuntimeFailures = new Map<
+  string,
+  Extract<PortablePodmanReadinessResult, { ok: false }>
+>();
 
 export type GatewayFailureLayer =
   | "docker_unreachable"
@@ -281,8 +290,22 @@ export function isDockerRuntimeDown(
   opts?: {
     runners?: Pick<GatewayFailureRunners, "dockerInfo">;
     getSandbox?: SandboxDriverLookup;
+    portableLifecycle?: PortableRuntimeReceiptReadinessDeps;
   },
 ): boolean {
+  const portable = inspectPortableRuntimeReceiptReadiness(sandboxName, opts?.portableLifecycle);
+  if (portable) {
+    if (portable.ok) {
+      portableRuntimeFailures.delete(sandboxName);
+      console.log(
+        `  Portable Podman readiness: ${portable.timing.mode}; activation ${String(portable.timing.activationMs)} ms; API ${String(portable.timing.apiMs)} ms; total ${String(portable.timing.totalMs)} ms.`,
+      );
+      return false;
+    }
+    portableRuntimeFailures.set(sandboxName, portable);
+    return true;
+  }
+  portableRuntimeFailures.delete(sandboxName);
   const getSandbox = opts?.getSandbox ?? registry.getSandbox;
   if (!isDockerBackedSandbox(sandboxName, getSandbox)) return false;
   const probe = opts?.runners?.dockerInfo ?? defaultRunners.dockerInfo;
@@ -302,6 +325,25 @@ export function printDockerRuntimeDownGuidance(
 ): void {
   const writer = opts.writer ?? console.error;
   const retryCommand = opts.retryCommand ?? "status";
+  const portable = portableRuntimeFailures.get(sandboxName);
+  portableRuntimeFailures.delete(sandboxName);
+  if (portable) {
+    writer(`  Failure stage: ${portable.stage} — ${portable.detail}`);
+    if (portable.socketPath) writer(`  Recorded socket: ${portable.socketPath}`);
+    writer(
+      `  Portable Podman readiness (${portable.timing.mode}): activation ${String(portable.timing.activationMs)} ms; API ${String(portable.timing.apiMs)} ms; total ${String(portable.timing.totalMs)} ms.`,
+    );
+    writer(
+      `  The receipt-owned Podman endpoint for sandbox '${sandboxName}' is not ready; no Docker or named-connection fallback was used.`,
+    );
+    writer("  Recovery:");
+    writer(
+      "    1. Check the reported readiness stage and the current user's Podman socket service.",
+    );
+    writer("    2. Confirm the recorded endpoint returns a real Podman server version.");
+    writer(`    3. Retry: ${CLI_NAME} ${sandboxName} ${retryCommand}`);
+    return;
+  }
   writer(`  ${getLayerHeader("docker_unreachable")}`);
   writer(
     `  The Docker daemon is not reachable, so sandbox '${sandboxName}' cannot be verified or started.`,

src/lib/onboard.ts

@@ -1518,11 +1518,9 @@ const { getSandboxRuntimeRegistryFields, hasSandboxGpuDrift, updateReusedSandbox
     getInstalledOpenshellVersion,
     runCaptureOpenshell,
   });
-
-// ── Step 5: Sandbox ──────────────────────────────────────────────
-
 async function createSandboxWithBaseImageResolution(
   baseImageResolutionContext: import("./onboard/base-image-resolution-flow").BaseImageResolutionContext,
+  portableRuntimeAuthority: import("./state/onboard-checkpoint-types").CheckpointPortableRuntimeAuthority | null,
   computePlan: import("./onboard/compute/plan").OpenShellComputePlan,
   managedWorkloadRebuild: import("./onboard/workload/rebuild").ManagedWorkloadRebuildHandoff | null,
   tempManagedRuntime: boolean,
@@ -1945,6 +1943,7 @@ async function createSandboxWithBaseImageResolution(
       sandboxEnv,
       sandboxStartupCommand,
       lifecycleGeneration: createdSandboxLifecycle.generation,
+      portableRuntimeAuthority,
       prebuild,
       restoreBackupPath,
       terminalAgent: agentDefs.isTerminalAgent(agent),
@@ -2111,6 +2110,7 @@ type CreateSandboxArgs =
     unknown,
     unknown,
     unknown,
+    unknown,
     ...infer Args,
   ]
     ? Args
@@ -2121,6 +2121,7 @@ const { createSandbox, createSandboxWithTemporaryManagedRuntime } =
     createBaseImageResolutionContext: () =>
       baseImageResolutionFlow.createBaseImageResolutionContext({ fresh: false }),
     createSandboxWithBaseImageResolution,
+    resolvePortableRuntimeAuthority: () => null,
     resolveComputePlan: dockerDriverPlatform.resolveCurrentOpenShellComputePlan,
   });
 
@@ -3556,6 +3557,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise<void> {
             withSandboxPortReservationScope((dashboardPortReservationScope) =>
               createSandboxWithBaseImageResolution(
                 baseImageResolutionContext,
+                lockedRuntime.preparedPortableAuthority,
                 onboardingComputePlan,
                 opts.managedWorkloadRebuild ?? null,
                 opts.tempManagedRuntime === true,

src/lib/onboard/dashboard-port.test.ts

@@ -261,6 +261,7 @@ describe("dashboard port reservation", () => {
     const createSandboxWithBaseImageResolution = vi.fn(
       async (
         baseImageResolutionContext: { fresh: boolean },
+        portableRuntimeAuthority: { socketPath: string } | null,
         computePlan: { sequence: number },
         managedWorkloadRebuild: null,
         temporaryManagedRuntime: boolean,
@@ -271,6 +272,7 @@ describe("dashboard port reservation", () => {
         events.push("create sandbox");
         return {
           baseImageResolutionContext,
+          portableRuntimeAuthority,
           computePlan,
           managedWorkloadRebuild,
           temporaryManagedRuntime,
@@ -287,6 +289,7 @@ describe("dashboard port reservation", () => {
         return { fresh: false };
       },
       createSandboxWithBaseImageResolution,
+      resolvePortableRuntimeAuthority: () => ({ socketPath: "/run/user/1001/podman.sock" }),
       resolveComputePlan: () => {
         events.push("resolve compute plan");
         return { sequence: ++sequence };
@@ -295,6 +298,7 @@ describe("dashboard port reservation", () => {
 
     await expect(entryPoints.createSandbox("standard")).resolves.toMatchObject({
       baseImageResolutionContext: { fresh: false },
+      portableRuntimeAuthority: { socketPath: "/run/user/1001/podman.sock" },
       computePlan: { sequence: 1 },
       managedWorkloadRebuild: null,
       temporaryManagedRuntime: false,
@@ -306,6 +310,7 @@ describe("dashboard port reservation", () => {
       entryPoints.createSandboxWithTemporaryManagedRuntime("temporary"),
     ).resolves.toMatchObject({
       baseImageResolutionContext: { fresh: false },
+      portableRuntimeAuthority: { socketPath: "/run/user/1001/podman.sock" },
       computePlan: { sequence: 2 },
       managedWorkloadRebuild: null,
       temporaryManagedRuntime: true,
@@ -314,8 +319,8 @@ describe("dashboard port reservation", () => {
       sandboxName: "temporary",
     });
     expect(createSandboxWithBaseImageResolution).toHaveBeenCalledTimes(2);
-    expect(createSandboxWithBaseImageResolution.mock.calls[0]?.[5]).not.toBe(
-      createSandboxWithBaseImageResolution.mock.calls[1]?.[5],
+    expect(createSandboxWithBaseImageResolution.mock.calls[0]?.[6]).not.toBe(
+      createSandboxWithBaseImageResolution.mock.calls[1]?.[6],
     );
     expect(events).toEqual([
       "resolve compute plan",
@@ -333,10 +338,12 @@ describe("dashboard port reservation", () => {
       [string],
       string,
       { fresh: boolean },
+      null,
       { sequence: number }
     >({
       createBaseImageResolutionContext: () => ({ fresh: false }),
       createSandboxWithBaseImageResolution: async () => "unreachable",
+      resolvePortableRuntimeAuthority: () => null,
       resolveComputePlan: () => {
         throw setupFailure;
       },