You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update Nemotron 3 Nano Omni documentation and layout for Jetson Thor support
- Added build URL for Nemotron 3 Nano Omni.
- Updated minimum Jetson model from Orin AGX to Thor.
- Removed unsupported AGX Orin commands and references from the documentation.
- Enhanced layout logic to manage module commands based on active inference engine.
- Introduced SGLang support in the inference engines configuration.
> **Note:**`--hf-repo ggml-org/NVIDIA-Nemotron-3-Nano-Omni` and `--hf-file nemotron-3-nano-omni-ga_v1.0-Q4_K_M.gguf` download the official GGUF checkpoint from Hugging Face. `-ngl 999` offloads all layers to GPU. `--alias my_model` sets the model name used in API requests. `chat_template_kwargs: {"enable_thinking": true}` activates chain-of-thought reasoning.
233
200
234
-
-->
235
-
236
201
## Running with TensorRT Edge-LLM
237
202
238
203
TensorRT Edge-LLM support for this model is currently Jetson Thor only. It requires exporting the model to ONNX and building TensorRT engines before running inference. See the [TensorRT Edge-LLM GitHub repository](https://github.com/NVIDIA/TensorRT-Edge-LLM) and the [export/build quick start](https://nvidia.github.io/TensorRT-Edge-LLM/user_guide/getting_started/quick-start-guide.html) for the full setup flow.
0 commit comments