Commit dbf48ba
authored
fix(inference): attach llama.cpp servers without /props model_alias (#9633)
<!-- markdownlint-disable MD041 -->
## Summary
Attaching an operator-run `llama-server` failed with `The server
returned conflicting or incomplete native llama.cpp evidence.` on every
build that omits `model_alias` from `/props`, because the attachment
probe required that field to equal the selected served alias. The probe
now requires the value to match only when the server returns the field,
so those servers attach, and the three fingerprint checks report
separately so the message names the endpoint that failed instead of
collapsing all three into one sentence.
## Related Issue
Fixes #9603
## Changes
- `src/lib/inference/llama-cpp/index.ts`: `hasMatchingNativeProps`
compares `/props` `model_alias` with the selected served alias only when
`/props` returns that field. `model_path`, a positive `total_slots`, and
`default_generation_settings.params` stay mandatory, and a returned
`model_alias` that differs from the served alias, including `null`, is
still refused.
- `src/lib/inference/llama-cpp/index.ts`: the combined health,
properties, and metrics check became three separate checks, each with a
message that names its endpoint and its missing evidence.
- `src/lib/inference/llama-cpp/index.test.ts`: five tests covering the
properties response without `model_alias`, the `null` alias, and one
message assertion for each endpoint.
- `docs/inference/set-up-llama-cpp.mdx`: states that the server needs an
explicit served model alias, and how NemoClaw uses `/props`
`model_alias` when the server returns it.
The tolerance for an absent `model_alias` is a compatibility path, so
its current requirement, consumer, and protecting test are:
- Current requirement: llama.cpp added `model_alias` to `/props` after
the builds reported in #9603. The revision pinned in
`managed-inference/recipes/llama-cpp.nemotron-3-nano-30b-a3b.spark-single.v1.yaml`
returns it; tag `b5400` does not, and returns every other field the
probe reads.
- Consumer: `probeLlamaCppAttachment`, reached from the attach path
through `src/lib/onboard/llama-cpp-selection/index.ts` and from managed
readiness through `src/lib/inference/llama-cpp/managed-status.ts`.
- Why a direct change is insufficient: the attach path registers a
server that NemoClaw does not own or version, so the probe cannot assume
the field is present. Removing the comparison outright would drop the
corroboration for servers that do return it, including a router that
answers `/props?model=` from the addressed instance.
- Protecting test: `attaches a native server whose properties omit the
served model alias (#9603)` and `rejects properties that report a null
served model alias (#9603)` in
`src/lib/inference/llama-cpp/index.test.ts`.
## Type of Change
- [ ] Code change (feature, bug fix, or refactor)
- [x] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)
## Quality Gates
<!-- Check one tests line. Check other lines when applicable. Add every
requested justification or approval reference. -->
- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:
## DGX Station Hardware Evidence
<!-- Required only when scripts/prepare-dgx-station-host.sh changes.
Maintainers must review the linked evidence before approving or merging.
This is human-reviewed evidence, not authenticated hardware provenance.
Exceptional bypasses use existing repository governance and must be
documented on the PR. -->
- [ ] Tested on DGX Station
- Tested commit:
- Station profile/scenario:
- Result:
- Supporting evidence:
## Verification
<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: `npx
vitest run --project cli src/lib/inference/llama-cpp/
src/lib/onboard/llama-cpp-selection/index.test.ts` — 9 files, 142
passed. `npm run typecheck:cli` — 0 errors. `npm run docs` — fern check
reports 0 errors and 2 warnings that main already reports.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved llama.cpp server attachment validation when model metadata is
missing or incomplete.
- Added clearer diagnostics for mismatched model aliases, invalid paths,
unavailable health responses, and missing metrics.
- Servers can now be recognized using other valid native indicators when
optional model alias metadata is omitted.
- **Documentation**
- Updated llama.cpp setup instructions with explicit model alias
requirements and metadata validation guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>1 parent fefc93e commit dbf48ba
3 files changed
Lines changed: 104 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
40 | 43 | | |
41 | 44 | | |
42 | 45 | | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
43 | 51 | | |
44 | 52 | | |
45 | 53 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
348 | 348 | | |
349 | 349 | | |
350 | 350 | | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
351 | 431 | | |
352 | 432 | | |
353 | 433 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
122 | 122 | | |
123 | 123 | | |
124 | 124 | | |
125 | | - | |
| 125 | + | |
| 126 | + | |
126 | 127 | | |
127 | 128 | | |
128 | 129 | | |
| |||
348 | 349 | | |
349 | 350 | | |
350 | 351 | | |
351 | | - | |
352 | | - | |
353 | | - | |
354 | | - | |
355 | | - | |
| 352 | + | |
356 | 353 | | |
357 | 354 | | |
358 | | - | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
359 | 368 | | |
360 | 369 | | |
361 | 370 | | |
| |||
0 commit comments