Skip to content

Commit fefc93e

Browse files
authored
fix(onboard): run portable Ollama under Podman (#9632)
<!-- markdownlint-disable MD041 --> ## Summary Fresh Hermes Portable onboarding now creates and proves a receipt-owned Ollama runner under the current user's rootless Podman runtime before provider selection completes. Previously the production resolver returned no host-local inference selection and fell through to the disabled host-Ollama readiness path. The default Docker daemon and host Ollama remain outside product authority. ## Related Issue Fixes #9596. ## Changes - Add the Hermes Portable Ollama authority owner used by fresh onboarding. It binds the prepared Podman executable and socket, Portable network, immutable image digest, exact GPU/CDI identity, model, and lifecycle receipt. A direct host-Ollama start is insufficient because Portable requires the current-user Podman runtime to own inference. - Add managed Ollama startup, readiness, model placement, route publication, recovery, and exact rollback through the existing host-local inference provider path. Focused provider and routing tests protect the accepted and rejected authority states. - Journal Gateway provider creation before mutation so onboarding can resume exact owned crash residue without adopting foreign or ambiguous providers. - Document the Hermes Portable Ollama command, runtime ownership, and the current schema-5 uninstall limitation tracked separately by #9608. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent exact-commit nine-category security review passed with no findings on `e9377da0cdce9d4678af92ef18f5e0aae70c5556` and manifest `e62b7fd3100633a7bd47d169b07edde0266ba41305d10af1e12c2cbbc8b20fb1`. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: the nine-file CLI suite passed 289/289 and `npx vitest run --project integration test/onboard-host-local-inference-routing.test.ts` passed 53/53. CLI build, CLI typecheck, repository checks, source-shape checks, growth checks, Oxlint, and diff checks passed on the exact commit. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: not run; the issue-scoped provider, routing, and production-composition suites exercise the changed runtime boundaries. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — `npm run docs` passed with zero errors and two existing Fern warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) — independent documentation writer review passed with no findings on the exact commit. - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added experimental portable Ollama onboarding using rootless Podman on Linux. - Added managed Ollama setup with explicit model selection, GPU support, health checks, and inference validation. - Added recovery for interrupted setup and durable provider configuration. - Added safeguards detecting changes to runtime, network, registry, GPU, listener, and model authority. - **Documentation** - Added setup and quickstart guidance, including preflight checks and cleanup limitations. - **Tests** - Added comprehensive coverage for setup, recovery, validation, rollback, listener changes, and provider configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
1 parent d34dab5 commit fefc93e

25 files changed

Lines changed: 4076 additions & 180 deletions

docs/get-started/quickstart-hermes.mdx

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,26 @@ Use these details when your first-run path needs more control.
130130
Refer to [Previous onboarding session failed](../reference/troubleshooting#previous-onboarding-session-failed) for recovery details.
131131
</Accordion>
132132

133+
<Accordion title="Use Portable Ollama">
134+
On Linux, fresh Hermes Portable onboarding can run the selected Ollama model in a current-user rootless Podman container.
135+
This path requires the Portable preflight to accept the current user's Podman and NVIDIA GPU authority.
136+
Start fresh onboarding with an explicit Ollama model:
137+
138+
```bash
139+
NEMOCLAW_PROVIDER=ollama \
140+
NEMOCLAW_MODEL=qwen3-vl:4b \
141+
nemohermes onboard --experimental-profile portable --fresh
142+
```
143+
144+
This path does not inspect, start, or use a host Ollama process.
145+
It does not use the default Docker runtime.
146+
Before the provider-selection step completes, NemoClaw creates the receipt-owned runner, sends one validation request for the selected model, and confirms that the exact model is loaded.
147+
NemoClaw records the provider selection only after those checks pass.
148+
149+
This onboarding path does not establish complete `destroy` or `uninstall` cleanup for the Portable Ollama runner.
150+
Refer to [Set Up Ollama](../inference/local-inference/set-up-ollama#use-portable-ollama-with-hermes) for the fail-closed uninstall boundary.
151+
</Accordion>
152+
133153
<Accordion title="Use the dashboard and API remotely">
134154
Hermes forwards its dashboard on port `18789` and its OpenAI-compatible API on port `8642`.
135155
A sandbox receives those ports when no other sandbox or host listener already holds them.

docs/inference/set-up-ollama.mdx

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,29 @@ NemoClaw detects Ollama on the host and can install, start, or upgrade it on sup
1818
## Prerequisites
1919

2020
- Install NemoClaw by following the [Quickstart](../../get-started/quickstart).
21-
- Use a host where Ollama is running or where the onboard wizard can install or start it.
21+
- For the host Ollama path, use a host where Ollama is running or where the onboard wizard can install or start it.
22+
23+
<AgentOnly variant="hermes">
24+
25+
## Use Portable Ollama with Hermes
26+
27+
Fresh Hermes Portable onboarding does not use the host Ollama setup described in the remaining sections.
28+
It creates a receipt-owned Ollama runner through the current user's rootless Podman authority.
29+
Run the fresh Portable command in the [Hermes quickstart](../../get-started/quickstart#use-portable-ollama) with `NEMOCLAW_PROVIDER=ollama` and an explicit `NEMOCLAW_MODEL`.
30+
31+
Before the provider-selection step completes, NemoClaw verifies the Portable network, registry, GPU, and Podman authority.
32+
It then creates the runner, sends one validation request for the selected model, and confirms that the exact model is loaded.
33+
This path does not inspect, start, or use a host Ollama process, and it does not use the default Docker runtime.
34+
35+
<Warning title="Portable Cleanup Boundary">
36+
This path covers fresh onboarding and provider selection.
37+
It does not establish complete `destroy` or `uninstall` cleanup for the Portable Ollama runner.
38+
Portable uninstall stops when it cannot prove exact cleanup authority for a Portable lifecycle receipt with schema `5`.
39+
It exits nonzero before deleting the runner or its receipt.
40+
Preserve the Portable Ollama runner and its lifecycle receipt.
41+
</Warning>
42+
43+
</AgentOnly>
2244

2345
## Install or Upgrade Ollama
2446

src/lib/onboard.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3179,7 +3179,7 @@ async function runOnboard(opts: OnboardOptions = {}): Promise<void> {
31793179
recoverySessionId,
31803180
),
31813181
setupInference,
3182-
resolveHostLocalInferenceStartupSelection: () => null,
3182+
resolveHostLocalInferenceStartupSelection: setupNimFlow.createHermesPortableOllamaInferenceResolver({ runtimeContext: lockedRuntime.portableRuntimeContext, credentialEnv: OLLAMA_PROXY_CREDENTIAL_ENV, getReservationSessionId: () => session?.sessionId, runGatewayOpenshell: runCoreGatewayOpenshell }),
31833183
startRecordedStep,
31843184
recordStepComplete,
31853185
recordStepRejected,

0 commit comments

Comments
 (0)