Merge pull request #387 from NVIDIA-AI-IOT/fix/llama

Update reasoning model documentation for Ministral 3 variants and Nemotron 3 Nano Omni

Author: adsahu-nv

src/content/models/ministral3-14b-reasoning.md

@@ -36,30 +36,11 @@ serving:
           --runtime=nvidia --network host \
           ghcr.io/nvidia-ai-iot/vllm:latest-jetson-thor \
           vllm serve mistralai/Ministral-3-14B-Reasoning-2512
-    - engine: "Ollama"
-      type: "CLI"
-      modules_supported:
-        - thor_t5000
-        - thor_t4000
-        - orin_agx_64
-      install_command: |-
-        curl -fsSL https://ollama.ai/install.sh | sh
-      serve_command_orin: ollama pull ministral-3:14b && ollama serve
-      serve_command_thor: ollama pull ministral-3:14b && ollama serve
-one_shot_inference:
-  modules_supported:
-    - thor_t5000
-    - thor_t4000
-    - orin_agx_64
-    - orin_nx_16
-    - orin_nano_8
-  run_command_orin: ollama run ministral-3:14b
-  run_command_thor: ollama run ministral-3:14b
 ---
 
 Mistral AI's Ministral 3 14B Reasoning is the most powerful reasoning variant, excelling at complex logical analysis and problem-solving.
 
-For **Ollama**, use the [`ministral-3:14b`](https://ollama.com/library/ministral-3) tag from the official library (check the library readme for minimum Ollama version).
+> **Note:** Ollama does not currently publish a separate tag for the Reasoning variant. The [`ministral-3:14b`](https://ollama.com/library/ministral-3) tag maps to the Instruct checkpoint. Use vLLM with the `mistralai/Ministral-3-14B-Reasoning-2512` HuggingFace checkpoint for the Reasoning model.
 
 The Ministral 3 Reasoning model offers the following capabilities:
 

src/content/models/ministral3-3b-reasoning.md

@@ -38,32 +38,11 @@ serving:
           --runtime=nvidia --network host \
           ghcr.io/nvidia-ai-iot/vllm:latest-jetson-thor \
           vllm serve mistralai/Ministral-3-3B-Reasoning-2512
-    - engine: "Ollama"
-      type: "CLI"
-      modules_supported:
-        - thor_t5000
-        - thor_t4000
-        - orin_agx_64
-        - orin_nx_16
-        - orin_nano_8
-      install_command: |-
-        curl -fsSL https://ollama.ai/install.sh | sh
-      serve_command_orin: ollama pull ministral-3:3b && ollama serve
-      serve_command_thor: ollama pull ministral-3:3b && ollama serve
-one_shot_inference:
-  modules_supported:
-    - thor_t5000
-    - thor_t4000
-    - orin_agx_64
-    - orin_nx_16
-    - orin_nano_8
-  run_command_orin: ollama run ministral-3:3b
-  run_command_thor: ollama run ministral-3:3b
 ---
 
 Mistral AI's Ministral 3 3B Reasoning is specifically optimized for logical reasoning, problem-solving, and analytical tasks.
 
-For **Ollama**, use the [`ministral-3:3b`](https://ollama.com/library/ministral-3) tag from the official library (check the library readme for minimum Ollama version).
+> **Note:** Ollama does not currently publish a separate tag for the Reasoning variant. The [`ministral-3:3b`](https://ollama.com/library/ministral-3) tag maps to the Instruct checkpoint. Use vLLM with the `mistralai/Ministral-3-3B-Reasoning-2512` HuggingFace checkpoint for the Reasoning model.
 
 The Ministral 3 Reasoning model offers the following capabilities:
 

src/content/models/ministral3-8b-reasoning.md

@@ -37,31 +37,11 @@ serving:
           --runtime=nvidia --network host \
           ghcr.io/nvidia-ai-iot/vllm:latest-jetson-thor \
           vllm serve mistralai/Ministral-3-8B-Reasoning-2512
-    - engine: "Ollama"
-      type: "CLI"
-      modules_supported:
-        - thor_t5000
-        - thor_t4000
-        - orin_agx_64
-        - orin_nx_16
-      install_command: |-
-        curl -fsSL https://ollama.ai/install.sh | sh
-      serve_command_orin: ollama pull ministral-3:8b && ollama serve
-      serve_command_thor: ollama pull ministral-3:8b && ollama serve
-one_shot_inference:
-  modules_supported:
-    - thor_t5000
-    - thor_t4000
-    - orin_agx_64
-    - orin_nx_16
-    - orin_nano_8
-  run_command_orin: ollama run ministral-3:8b
-  run_command_thor: ollama run ministral-3:8b
 ---
 
 Mistral AI's Ministral 3 8B Reasoning is the default reasoning variant, balancing reasoning capability with efficiency.
 
-For **Ollama**, use the [`ministral-3:8b`](https://ollama.com/library/ministral-3) tag from the official library (check the library readme for minimum Ollama version).
+> **Note:** Ollama does not currently publish a separate tag for the Reasoning variant. The [`ministral-3:8b`](https://ollama.com/library/ministral-3) tag maps to the Instruct checkpoint. Use vLLM with the `mistralai/Ministral-3-8B-Reasoning-2512` HuggingFace checkpoint for the Reasoning model.
 
 The Ministral 3 Reasoning model offers the following capabilities:
 

src/content/models/nemotron-3-nano-omni.md

@@ -64,10 +64,14 @@ supported_inference_engines:
     serve_command_thor: |-
       sudo docker run -it --rm --pull always \
         --runtime=nvidia --network host \
-        -v $HOME/.cache/huggingface:/data/models/huggingface \
-        -v $HOME/.cache/llama.cpp:/data/models/llama.cpp \
         ghcr.io/nvidia-ai-iot/llama_cpp:latest-jetson-thor \
-        bash -lc 'MODEL=$(huggingface-downloader ggml-org/Nemotron-3-Nano-Omni-GGUF/nemotron-3-nano-omni-ga_v1.0-Q4_K_M.gguf) && MMPROJ=$(huggingface-downloader ggml-org/Nemotron-3-Nano-Omni-GGUF/mmproj-nemotron-3-nano-omni-ga_v1.0.gguf) && llama-server -m "$MODEL" -mm "$MMPROJ" --ctx-size 8192 --alias my_model --n-gpu-layers 999 --host 0.0.0.0 --port 8080'
+        llama-server \
+          --hf-repo ggml-org/Nemotron-3-Nano-Omni-GGUF \
+          --hf-file nemotron-3-nano-omni-ga_v1.0-Q4_K_M.gguf \
+          --ctx-size 8192 \
+          --port 8080 \
+          --alias my_model \
+          --n-gpu-layers 999
   - engine: "Ollama"
     type: "Local"
     modules_supported:
@@ -168,17 +172,14 @@ curl -s http://${JETSON_HOST}:30000/v1/chat/completions \
 ```bash
 sudo docker run -it --rm --pull always \
   --runtime=nvidia --network host \
-  -v $HOME/.cache/huggingface:/data/models/huggingface \
-  -v $HOME/.cache/llama.cpp:/data/models/llama.cpp \
   ghcr.io/nvidia-ai-iot/llama_cpp:latest-jetson-thor \
-  bash -lc 'MODEL=$(huggingface-downloader ggml-org/Nemotron-3-Nano-Omni-GGUF/nemotron-3-nano-omni-ga_v1.0-Q4_K_M.gguf) && \
-    MMPROJ=$(huggingface-downloader ggml-org/Nemotron-3-Nano-Omni-GGUF/mmproj-nemotron-3-nano-omni-ga_v1.0.gguf) && \
-    llama-server -m "$MODEL" -mm "$MMPROJ" \
-      --ctx-size 8192 \
-      --alias my_model \
-      --n-gpu-layers 999 \
-      --host 0.0.0.0 \
-      --port 8080'
+  llama-server \
+    --hf-repo ggml-org/Nemotron-3-Nano-Omni-GGUF \
+    --hf-file nemotron-3-nano-omni-ga_v1.0-Q4_K_M.gguf \
+    --ctx-size 8192 \
+    --port 8080 \
+    --alias my_model \
+    --n-gpu-layers 999
 ```
 
 </div>
@@ -199,7 +200,7 @@ curl -s http://127.0.0.1:8080/v1/chat/completions \
   }'
 ```
 
-> **Note:** `huggingface-downloader` fetches the GGUF model and multimodal projector from `ggml-org/Nemotron-3-Nano-Omni-GGUF`. `-mm "$MMPROJ"` loads the multimodal projector for vision support. `--n-gpu-layers 999` offloads all layers to GPU. `--alias my_model` sets the model name used in API requests. `chat_template_kwargs: {"enable_thinking": true}` activates chain-of-thought reasoning.
+> **Note:** `--hf-repo ggml-org/Nemotron-3-Nano-Omni-GGUF` and `--hf-file nemotron-3-nano-omni-ga_v1.0-Q4_K_M.gguf` download the official GGUF checkpoint from Hugging Face. `--n-gpu-layers 999` offloads all layers to GPU. `--alias my_model` sets the model name used in API requests. `chat_template_kwargs: {"enable_thinking": true}` activates chain-of-thought reasoning.
 
 ## Running with Ollama