Skip to content

Commit 522b79d

Browse files
authored
docs(troubleshooting): correct the GPU-reconnect debounce default (#9536)
## Summary The GPU compatibility troubleshooting section documented the OpenShell supervisor-reconnect debounce as fifteen polls of about 30 seconds, and told a stuck operator to raise it "to a higher integer such as `30`". The current default is 60 polls of about 120 seconds, so `30` is half the default: an operator who followed the page shortened the debounce and made the reconnect failure the section describes more likely. This change states the current default and recommends a value above it. ## Related Issue Fixes #9535 ## Changes - `docs/reference/troubleshooting.mdx` — state the reconnect debounce default as 60 consecutive polls of about 120 seconds, matching `DOCKER_GPU_SUPERVISOR_RECONNECT_ERROR_PHASE_DEFAULT_DEBOUNCE_POLLS = 60` at `src/lib/onboard/docker-gpu-supervisor-reconnect.ts:50` and the `sleep(2)` poll interval at `:139`. - `docs/reference/troubleshooting.mdx` — replace the recommended value `30` with `120` (about 240 seconds) and name the default in the same sentence, so the recommended value is above the default rather than below it. Both edits rewrite existing sentences. The diff is 2 insertions and 2 deletions, net 0 lines. No code, no test, and no new page. ### What an operator should actually set, and why Set `NEMOCLAW_DOCKER_GPU_SUPERVISOR_RECONNECT_ERROR_DEBOUNCE` to an integer above 60. The page now uses `120`, which is about 240 seconds of sustained Error phase before fast-fail. `120` is a usable recommendation rather than an arbitrary one because the debounce is bounded by a separate deadline. `getDockerGpuSupervisorReconnectTimeoutSecs` (`src/lib/onboard/docker-gpu-supervisor-reconnect.ts:144-153`) floors the reconnect wait at `DOCKER_GPU_SUPERVISOR_RECONNECT_MIN_SECS = 900` seconds (`:30`). A 240-second debounce therefore stays well inside the surrounding wait and cannot itself cause the timeout it is meant to avoid. Doubling the default is also the shape the code comment at `:31-36` describes: the default already exists to give Docker-CDI GPU runners headroom past the original ~30 second window, and a slow WSL2 or Docker Desktop host needs more of the same headroom, not less. ### How the page drifted `8827570b9` ("fix(onboard): prefer CDI GPU mode over --gpus on CDI hosts", #4956) raised the constant from `15` to `60` and changed no file under `docs/`. The prose dates from `43efeba74` (#4777), which introduced the `15` default and this section together. The existing test `src/lib/onboard/docker-gpu-supervisor-reconnect.test.ts:171` pins the default at 60, and `:222-223` pins the 60-polls-plus-59-sleeps shape of the window, so the documented value was measurably wrong rather than debatable. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [x] Existing tests cover changed behavior — justification: `src/lib/onboard/docker-gpu-supervisor-reconnect.test.ts:171-172` already asserts `getDockerGpuSupervisorReconnectErrorDebouncePolls({})` is `60`, and `:222-223` asserts 60 polls and 59 sleeps for the default window. The corrected prose restates those pinned values. No product behavior changes in this PR. - [ ] Tests not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: This PR describes an onboarding code path but changes only `docs/reference/troubleshooting.mdx`, so no sensitive path is modified. ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable — `npm run validate:pr` exit 0 on this branch; `npx commitlint --from origin/main --to HEAD` exit 0. - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — justification: documentation-only change; the values it states are pinned by the existing `docker-gpu-supervisor-reconnect.test.ts` cases named in Quality Gates. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: not applicable, no runtime, test-harness, or repo-wide validation change. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [x] `npm run docs` builds without warnings (doc changes only) — `Found 0 errors and 2 warnings`, and `check-docs-published-routes: OK — 68 guarded page(s)`. Both warnings are pre-existing and unrelated to this change: the Fern redirect check is skipped because the local run is unauthenticated, and the site accent contrast ratio is a theme setting. Both also appear on an unmodified checkout of `main`. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated troubleshooting guidance to reflect a longer default GPU supervisor reconnect delay. * Updated the example override value; settings remain integer-based with a minimum of 1. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com>
1 parent 8262422 commit 522b79d

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

docs/reference/troubleshooting.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3211,9 +3211,9 @@ This injection happens inside the gateway, so `NEMOCLAW_DOCKER_GPU_PATCH=0` does
32113211
Rerun with `--no-gpu`, or set `NEMOCLAW_SANDBOX_GPU=0` and resume onboarding.
32123212

32133213
If onboarding reports `OpenShell supervisor did not reconnect to the GPU-enabled container.` even though the diagnostic bundle shows the patched container is running and healthy, the supervisor-reconnect wait is treating a transient Error phase (reported while the OpenShell host re-registers the new container) as fatal.
3214-
The reconnect wait debounces consecutive Error-phase polls before fast-failing, defaulting to fifteen consecutive polls of about 30 seconds in total.
3214+
The reconnect wait debounces consecutive Error-phase polls before fast-failing, defaulting to 60 consecutive polls of about 120 seconds in total.
32153215
Increase the debounce window with `NEMOCLAW_DOCKER_GPU_SUPERVISOR_RECONNECT_ERROR_DEBOUNCE` if your host needs more time to re-register the patched container, for example slow WSL2 + Docker Desktop setups.
3216-
Set it to a higher integer such as `30` (about 60 seconds) and rerun onboarding; the value is clamped to a minimum of `1`.
3216+
Set it to an integer above the default of 60, such as `120` (about 240 seconds), and rerun onboarding; the value is clamped to a minimum of `1`.
32173217
If reconnect still fails after the GPU patch, NemoClaw attempts to restore the pre-patch CPU container before exiting.
32183218
When rollback succeeds, the output says the pre-patch sandbox was restored.
32193219
When rollback fails, the error says rollback failed and the pre-patch container was not restored, so inspect Docker state before retrying.

0 commit comments

Comments
 (0)