Skip to content

Commit a222880

Browse files
committed
feat(inference): add Linux Muse and Lightning profiles
Add explicit-only Linux amd64 catalog presets and recipes for Muse Glimmer 30B. Add a matching Nemotron 3.5 Lightning profile. Cover direct selection, architecture and memory rejection, runtime floors, and Spark precedence. Refs #9673 Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
1 parent ae4bd18 commit a222880

90 files changed

Lines changed: 487 additions & 1474 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/skills/_shared/code-change-considerations.md

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -17,22 +17,13 @@ recording them here.
1717

1818
- What accepted outcome and current consumer require the change?
1919
- What current code owns the behavior, and can that owner be extended directly?
20-
- What current code, branch, parameter, owner, fixture, or file becomes unnecessary and can be
21-
deleted or merged in this change?
2220
- Would the change duplicate an existing structure or create another source of truth?
23-
- Can the completed source-and-test change be neutral or negative in total lines? If not, what
24-
current correctness, security, or accepted-scope contract requires the growth?
25-
- If the change adds a helper, abstraction, configuration, registry, fallback, or compatibility
26-
path, which current consumers adopt it now, what old structure does it remove, and is the whole
27-
result smaller or simpler?
2821
- What state, success, failure, and partial-failure behavior must remain coherent?
2922
- What ordering or concurrency can change the result or bypass a guarantee?
3023
- How do absent values, defaults, retries, recovery, and cleanup behave?
3124
- Which alternate entry, error, cached, resumed, or compatibility paths can bypass the change?
3225
- Can code or configuration be removed, or can an existing or native mechanism replace new code?
3326
- What shortest stable test proves the changed behavior, including the relevant negative path?
34-
- Can that evidence extend or consolidate current fixtures, matrices, and assertions instead of
35-
creating another test owner or a one-use test helper?
3627
- Does a real process, network, filesystem, container, hardware, or service boundary require deeper
3728
runtime or end-to-end evidence?
3829
- Which active issues, pull requests, or recent changes overlap, conflict, or affect delivery order?

.agents/skills/nemoclaw-contributor-implement-issue/SKILL.md

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -80,25 +80,13 @@ checked for that operation or failure class and the sensitive-workflow state out
8080
hold. Then make the direct change in the current behavior owner. Do not add speculative abstractions,
8181
configuration, compatibility, migration, or fallback behavior.
8282

83-
Optimize the complete source-and-test change for deletion and consolidation. Compare a direct edit,
84-
reuse of an existing owner, and a refactor of current related code. Prefer a neutral or negative
85-
total line delta. Add a helper, abstraction, configuration surface, registry, fallback, or
86-
compatibility path only when current consumers adopt it in this change and the complete result
87-
removes more owners, concepts, branches, or lines than it adds. Possible future reuse is not enough.
88-
When a current correctness, security, or accepted-scope contract requires growth, keep the design
89-
direct and record why deletion or reuse cannot satisfy it.
90-
9183
Add focused evidence as applicable:
9284

9385
- positive behavior that must succeed;
9486
- negative or denied behavior that must fail;
9587
- error, interruption, recovery, or cleanup behavior;
9688
- boundary values, ambiguous state, and alternate entry paths.
9789

98-
Preserve semantic regression coverage, not every existing fixture or assertion block. Extend or
99-
table-drive current coverage when that keeps one setup and one behavior owner. Do not create a
100-
one-use test helper, parallel matrix, or second test file merely to shorten an individual test.
101-
10290
State why an evidence category does not apply when omission could hide risk. Use runtime or end-to-end
10391
evidence only when the real process, filesystem, network, container, hardware, workflow, or service
10492
boundary owns the behavior. Run focused tests after the final behavior-affecting edit and record the
@@ -115,11 +103,6 @@ Consideration and all nine Security Rubric categories to the completed behavior.
115103
security control and focused negative evidence that proves forbidden behavior remains denied. If no
116104
security control changed, state why and cite the reviewed trust boundaries.
117105

118-
Record the reduction case for the completed design: the current code, owners, branches, parameters,
119-
fixtures, or files deleted or consolidated and the total source-and-test line direction. If the
120-
change grew, identify the current contract that requires that growth. Remove review-driven machinery
121-
when a direct solution is smaller; do not add another layer to compensate for an avoidable layer.
122-
123106
Confirm that allowed, denied, error, and boundary behavior remains coherent across failure, retry,
124107
cleanup, cached, resumed, and compatibility paths that apply.
125108
Re-check the recorded operation and failure class, sibling paths, and sensitive-workflow state
@@ -138,7 +121,6 @@ Use this structure:
138121
- Accepted scope authority:
139122
- Delivered capability:
140123
- Changed behavior:
141-
- Simplification result: <deleted or consolidated structure, total line direction, and required-growth justification when applicable>
142124
- Deferred scope:
143125

144126
## Changed files

.github/workflows/podman-cpu-proof.yaml

Lines changed: 0 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,6 @@ on:
2828
- "src/lib/onboard/runtime-provider/container-state-mutation.ts"
2929
- "src/lib/onboard/runtime-provider/docker-state-mutation.ts"
3030
- "src/lib/onboard/experimental/portable-host-preparation*.ts"
31-
- "src/lib/onboard/experimental/portable-profile.ts"
3231
- "src/lib/onboard/runtime-provider/podman*.ts"
3332
- "scripts/install-openshell.sh"
3433
- "scripts/checks/run-portable-cpu-delegation-proof.mts"
@@ -335,21 +334,6 @@ jobs:
335334
printf 'XDG_RUNTIME_DIR=%s\n' "$runtime_dir"
336335
} >>"$GITHUB_ENV"
337336
338-
- name: Configure exact Portable host gateway alias
339-
shell: bash
340-
run: |
341-
set -euo pipefail
342-
portable_host_gateway_ip="$(
343-
node --input-type=module --eval '
344-
const { PORTABLE_HOST_GATEWAY_IP } =
345-
await import("./dist/lib/onboard/docker-driver-platform.js");
346-
process.stdout.write(PORTABLE_HOST_GATEWAY_IP);
347-
'
348-
)"
349-
sudo ip address replace "$portable_host_gateway_ip/32" dev lo
350-
ip -o -4 address show dev lo | awk '{print $4}' | grep -Fx "$portable_host_gateway_ip/32"
351-
printf 'E2E_PORTABLE_HOST_GATEWAY_IP=%s\n' "$portable_host_gateway_ip" >>"$GITHUB_ENV"
352-
353337
- name: Start the exact connected gateway required by portable retirement
354338
shell: bash
355339
run: |
@@ -474,17 +458,6 @@ jobs:
474458
esac
475459
done < <(podman --url "$endpoint" secret ls --format '{{.Name}}' 2>/dev/null || true)
476460
podman --url "$endpoint" network rm openshell-docker 2>/dev/null || true
477-
portable_host_gateway_ip=""
478-
if portable_host_gateway_ip="$(
479-
node --input-type=module --eval '
480-
const { PORTABLE_HOST_GATEWAY_IP } =
481-
await import("./dist/lib/onboard/docker-driver-platform.js");
482-
process.stdout.write(PORTABLE_HOST_GATEWAY_IP);
483-
'
484-
)" && [ -n "$portable_host_gateway_ip" ] \
485-
&& [ "${E2E_PORTABLE_HOST_GATEWAY_IP:-}" = "$portable_host_gateway_ip" ]; then
486-
sudo ip address delete "$portable_host_gateway_ip/32" dev lo 2>/dev/null || true
487-
fi
488461
service_pid="${E2E_PODMAN_SERVICE_PID:-}"
489462
if [[ "$service_pid" =~ ^[1-9][0-9]*$ ]]; then
490463
kill "$service_pid" 2>/dev/null || true

bin/nemoclaw.js

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,9 @@ try {
1717
}
1818

1919
const PORT_ENV_NAME =
20-
"NEMOCLAW_(?:GATEWAY|DASHBOARD|VLLM|LLAMACPP|OLLAMA|OLLAMA_PROXY|BEDROCK_RUNTIME_ADAPTER|OPENROUTER_RUNTIME_ADAPTER|HTTPS_PIN_RUNTIME_ADAPTER)_PORT";
20+
"NEMOCLAW_(?:GATEWAY|DASHBOARD|VLLM|OLLAMA|OLLAMA_PROXY|BEDROCK_RUNTIME_ADAPTER|OPENROUTER_RUNTIME_ADAPTER|HTTPS_PIN_RUNTIME_ADAPTER)_PORT";
2121
const SAFE_PORT_DIAGNOSTIC = new RegExp(
22-
`^Invalid port: ${PORT_ENV_NAME}="\\d{1,5}" — (?:must be an integer between 1024 and 65535|must not overlap the 18789-18799 dashboard port range|must not overlap the (?:llama\\.cpp inference|vLLM / NIM inference|Ollama inference|Ollama auth proxy|Bedrock Runtime adapter|OpenRouter Runtime adapter|HTTPS Pin Runtime adapter) default port \\(\\d{1,5}\\)|conflicts with ${PORT_ENV_NAME} \\(\\d{1,5}\\))$`,
22+
`^Invalid port: ${PORT_ENV_NAME}="\\d{1,5}" — (?:must be an integer between 1024 and 65535|must not overlap the 18789-18799 dashboard port range|must not overlap the (?:llama\\.cpp inference|vLLM / NIM inference|Ollama inference|Ollama auth proxy|Bedrock Runtime adapter|OpenRouter Runtime adapter|HTTPS Pin Runtime adapter) default port \\(\\d{1,5}\\)|conflicts with ${PORT_ENV_NAME} \\(\\d{1,5}\\)|conflicts with the fixed llama\\.cpp inference port \\(8081\\))$`,
2323
);
2424

2525
function redactFallbackMessage(message) {

docs/inference/choose-local-inference-server.mdx

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ Before you start, confirm these prerequisites:
8181

8282
- Use a DGX Spark host with Linux on Arm64.
8383
- Confirm that Docker, the NVIDIA Container Toolkit, and the NVIDIA GPU driver are operational.
84-
- Stop any server that already uses the selected `NEMOCLAW_VLLM_PORT` (`8000` by default).
84+
- Stop any server that already uses port `8000`.
8585
- Allow capacity for container images, model artifacts, and same-filesystem download staging.
8686
- Export `HF_TOKEN` only if the configured artifact source requires Hugging Face authentication.
8787

@@ -113,14 +113,13 @@ curl -fsSL https://www.nvidia.com/nemoclaw.sh | \
113113
The flag enables the dedicated vLLM profile gate, disables Express selection, and selects the matching catalog recipe.
114114
Selecting Express option 2 enables the same profile gate and selects the vLLM runtime.
115115
Do not combine this path with `NEMOCLAW_PROVIDER` or `NEMOCLAW_MODEL`.
116-
The profile rejects `NEMOCLAW_VLLM_MODEL` and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` before installation.
117-
Set `NEMOCLAW_VLLM_PORT` to publish its fixed container listener on another unused host port.
116+
The profile rejects `NEMOCLAW_VLLM_MODEL`, `NEMOCLAW_VLLM_PORT`, and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` before installation.
118117

119118
The profile performs these actions:
120119

121120
- Uses the pinned runtime image and fixed serving command from the catalog.
122121
- Stores model files in the host Hugging Face cache.
123-
- Publishes the authenticated server on loopback and the exact private IPv4 gateway of the `openshell-docker` bridge at the configured `NEMOCLAW_VLLM_PORT` (`8000` by default).
122+
- Publishes the authenticated server on `127.0.0.1:8000` and the exact private IPv4 gateway of the `openshell-docker` bridge, such as `172.18.0.1:8000`.
124123
- Never publishes the Docker host port on `0.0.0.0` or `::`.
125124
- Reuses the owner-only host-global managed-vLLM API key.
126125

@@ -149,7 +148,7 @@ docker container inspect \
149148
```
150149

151150
The bridge result must contain exactly one private IPv4 `Gateway` value.
152-
The container result must contain exactly two bindings with `HostPort` set to the selected `NEMOCLAW_VLLM_PORT` (`8000` by default): one `HostIp` value of `127.0.0.1` and one that exactly matches the private bridge gateway. The inspect key remains `8000/tcp` because the container listener is fixed even when the published host port changes.
151+
The container result must contain exactly two bindings with `HostPort` set to `8000`: one `HostIp` value of `127.0.0.1` and one that exactly matches the private bridge gateway.
153152
Reject the runtime if a binding uses `0.0.0.0`, `::`, an empty address, another host address, or an additional entry.
154153

155154
If installation stops after a model download, rerun the same installer command.

docs/inference/set-up-llama-cpp.mdx

Lines changed: 11 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Both paths are experimental and use OpenAI Chat Completions through `https://inf
1717

1818
| Path | NemoClaw manages | You manage | Use it when |
1919
|---|---|---|---|
20-
| Existing server | Provider registration, route validation, and sandbox traffic through `inference.local`. | The server binary or container, GGUF file, launch configuration, credential, upgrades, and lifecycle. | Your authenticated server already satisfies the llama.cpp fingerprint contract on the configured loopback port (`8081` by default). |
20+
| Existing server | Provider registration, route validation, and sandbox traffic through `inference.local`. | The server binary or container, GGUF file, launch configuration, credential, upgrades, and lifecycle. | Your authenticated server already satisfies the llama.cpp fingerprint contract on loopback port `8081`. |
2121
| Managed DGX Spark | The exact YAML-selected image, GGUF acquisition and verification, launch configuration, credential, Docker lifecycle, diagnostics, and cleanup. | The qualified Linux host, Docker Engine, NVIDIA runtime prerequisites, storage, and any required Hugging Face credential. | You want the experimental Muse Glimmer or NVIDIA Nemotron profile without managing `llama-server` directly. |
2222

2323
Compared with other local options, Ollama emphasizes a simple local model workflow, while managed vLLM provides host-specific NVIDIA GPU profiles and broader model selection.
@@ -30,7 +30,7 @@ Refer to [Choose a Local Inference Server](choose-local-inference-server) for th
3030
Use this path when you operate `llama-server` and want NemoClaw to register its authenticated endpoint without owning its process or model.
3131
The server must satisfy all of these requirements:
3232

33-
- Listen over HTTP on `127.0.0.1:<configured-port>` and expose its OpenAI-compatible API under `/v1`.
33+
- Listen over HTTP on `127.0.0.1:8081` and expose its OpenAI-compatible API under `/v1`.
3434
- Require the same native bearer key supplied through `NEMOCLAW_LLAMACPP_LOCAL_TOKEN`.
3535
- Expose bounded native llama.cpp evidence through `/v1/models`, `/health`, `/props`, and either `/metrics` or the native metrics-not-supported response.
3636
- Report a stable, non-path served model alias.
@@ -64,7 +64,6 @@ For non-interactive onboarding, export `NEMOCLAW_LLAMACPP_LOCAL_TOKEN` from your
6464
: "${NEMOCLAW_LLAMACPP_LOCAL_TOKEN:?Export the native llama.cpp API key first}"
6565
NEMOCLAW_PROVIDER=llama-cpp \
6666
NEMOCLAW_MODEL="<served-model-alias>" \
67-
NEMOCLAW_LLAMACPP_PORT=19081 \
6867
NEMOCLAW_SANDBOX_NAME=my-assistant \
6968
$$nemoclaw onboard --non-interactive --yes-i-accept-third-party-software
7069
```
@@ -74,10 +73,6 @@ You can omit `NEMOCLAW_MODEL` only when the server exposes exactly one native mo
7473
It refuses unauthenticated, ambiguous, non-native, conflicting, or differently addressed servers instead of guessing their identity.
7574
It does not start, stop, upgrade, or remove an attached server.
7675

77-
Set `NEMOCLAW_LLAMACPP_PORT` to the server's non-privileged loopback port.
78-
The variable defaults to `8081` and also controls the OpenShell gateway route and local-inference policy entry.
79-
For a managed runtime, NemoClaw persists the chosen bridge port and reuses it during later recovery.
80-
8176
Verify the registered route:
8277

8378
```bash
@@ -110,7 +105,7 @@ Before you start, confirm these prerequisites:
110105
- Confirm that the NVIDIA Container Toolkit is operational.
111106
- Confirm that NVIDIA Container Device Interface support is healthy.
112107
- Use NVIDIA driver version `580.65.06` or later.
113-
- Stop any process that uses the configured host port (`8081` when `NEMOCLAW_LLAMACPP_PORT` is unset).
108+
- Stop any process that uses host port `8081`.
114109
- Allow capacity for the pinned images, the GGUF file, and same-filesystem download staging.
115110
- Export `HF_TOKEN` only when the YAML-declared Hugging Face source requires authentication.
116111

@@ -121,7 +116,7 @@ Run `unset HF_TOKEN` after onboarding when no other process needs it.
121116
<Warning>
122117
When the exact artifacts are not already present, onboarding pulls digest-pinned images from external registries and acquires the YAML-declared GGUF file through the standard Hugging Face cache.
123118
It reuses locally present pinned images and a cached GGUF only after their exact identities verify.
124-
It creates an owner-only API key, a host-loopback listener on the configured port, and a Docker internal network.
119+
It creates an owner-only API key, a host-loopback listener on port `8081`, and a Docker internal network.
125120
Only one managed llama.cpp runtime can exist per Docker authority, regardless of the owning OpenShell gateway or sandbox.
126121
</Warning>
127122

@@ -143,7 +138,6 @@ For non-interactive onboarding, select the repository-owned recipe explicitly:
143138
```bash
144139
NEMOCLAW_PROVIDER=install-llama-cpp \
145140
NEMOCLAW_LLAMACPP_RECIPE=llama-cpp.muse-glimmer-30b.spark-single.v1 \
146-
NEMOCLAW_LLAMACPP_PORT=19081 \
147141
NEMOCLAW_SANDBOX_NAME=my-assistant \
148142
$$nemoclaw onboard --non-interactive --yes-i-accept-third-party-software
149143
```
@@ -157,8 +151,7 @@ For non-interactive onboarding or an explicit provider request, set `NEMOCLAW_LL
157151
An unknown recipe, an incompatible host, or more than one compatible profile for the requested recipe stops onboarding before installation effects.
158152
Do not set `NEMOCLAW_MODEL` for this path.
159153
The repository YAML recipe is authoritative for the exact model file, revision, digest, image, launch settings, resource limits, readiness probes, and disabled surfaces.
160-
`NEMOCLAW_LLAMACPP_PORT` controls the host-owned private bridge; the recipe's internal server port remains fixed.
161-
NemoClaw does not provide hidden model, image, or launch-flag overrides for the managed path.
154+
NemoClaw does not provide hidden model, port, image, or launch-flag overrides for the managed path.
162155

163156
The shipped NVIDIA Nemotron recipe continues to use the `nemotron-v3-embedded` chat template.
164157
A repository recipe that uses an image-owned Jinja template must declare this typed contract:
@@ -196,7 +189,7 @@ Onboarding performs these actions:
196189
- Reuses a verified GGUF from the shared `~/.cache/huggingface/` cache, or acquires the exact file through the existing Hugging Face mechanism when it is absent or invalid.
197190
- Verifies the GGUF size and SHA-256 digest before it starts the runtime.
198191
- Runs the authenticated container as the current non-root user with one NVIDIA GPU and no CPU fallback.
199-
- Starts a host-owned private bridge on `127.0.0.1:<configured-port>` and connects the container to a Docker internal network without a Docker-published port.
192+
- Starts a host-owned private bridge on `127.0.0.1:8081` and connects the container to a Docker internal network without a Docker-published port.
200193
- Registers `llama-cpp-local` and routes agent traffic through `https://inference.local/v1`.
201194

202195
The runtime cannot download a model and has no egress on its internal Docker network.
@@ -232,14 +225,14 @@ Refer to [CLI Commands](../../reference/commands) for complete command behavior.
232225

233226
During managed installation, NemoClaw first checks internal runtime readiness and host-loopback health.
234227
It repeats these checks when onboarding resumes.
235-
It then runs the authoritative OpenShell Docker bridge probe on the configured host port.
228+
It then runs the authoritative OpenShell Docker bridge probe on fixed port `8081`.
236229
If that bridge probe cannot connect or times out after the earlier checks pass, onboarding stops without changing UFW.
237230
When the bridge reports a valid narrow subnet and a gateway IP address inside that subnet, the error reports these values:
238231

239232
- The detected Docker network.
240233
- The source subnet.
241234
- The gateway IP address.
242-
- The configured port.
235+
- The fixed port `8081`.
243236
- The exact narrow UFW command.
244237

245238
If the bridge topology is missing or invalid, onboarding fails closed without printing topology or a UFW command.
@@ -248,22 +241,22 @@ Inspect the OpenShell Docker bridge configuration before you retry onboarding.
248241
<Warning>
249242
The printed command changes UFW.
250243
Confirm that the reported Docker network, source subnet, and gateway IP address belong to the OpenShell Docker bridge before you run it.
251-
The rule allows TCP traffic only from that subnet to the configured port on that gateway IP address.
244+
The rule allows TCP traffic only from that subnet to port `8081` on that gateway IP address.
252245
</Warning>
253246

254247
Run the exact command from the onboarding error.
255248
It has this form:
256249

257250
```bash
258-
sudo ufw allow from <subnet> to <gateway-ip> port <llama-cpp-port> proto tcp
251+
sudo ufw allow from <subnet> to <gateway-ip> port 8081 proto tcp
259252
```
260253

261254
After you apply the rule, rerun the same onboarding selection.
262255

263256
If onboarding still reports the bridge failure, or when the managed runtime no longer needs the rule, remove the exact rule that you added:
264257

265258
```bash
266-
sudo ufw --force delete allow from <subnet> to <gateway-ip> port <llama-cpp-port> proto tcp
259+
sudo ufw --force delete allow from <subnet> to <gateway-ip> port 8081 proto tcp
267260
```
268261

269262
Use the same subnet and gateway IP address that appeared in the original command.

0 commit comments

Comments
 (0)