Add Voxtral-Mini-4B-Realtime model via vLLM backend

Add support for the Voxtral-Mini-4B-Realtime-2602 model from Mistral AI
using the vLLM backend. This enables high-throughput inference for
real-time speech-to-text transcription.

- Added new gallery config: gallery/voxtral-mini-4b-realtime-vllm.yaml
- Added model entry to gallery/index.yaml
- Enabled trust_remote_code for HuggingFace model loading

Closes: mudler#8401

Author: localai-bot

gallery/index.yaml

@@ -64,6 +64,30 @@
       - transcript
     parameters:
       model: nvidia/parakeet-tdt-0.6b-v3
+- name: voxtral-mini-4b-realtime-vllm
+  license: apache-2.0
+  url: "github:mudler/LocalAI/gallery/voxtral-mini-4b-realtime-vllm.yaml@master"
+  icon: https://huggingface.co/datasets/huggingface/brand-assets/resolve/main/hf-logo.svg
+  description: |
+    Voxtral Mini 4B Realtime via vLLM - A speech-to-text model from Mistral AI served using the vLLM backend. This 4B parameter model is optimized for fast, accurate audio transcription with low latency, making it ideal for real-time applications. Using vLLM enables high-throughput inference and efficient GPU utilization.
+  urls:
+    - https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602
+  tags:
+    - stt
+    - speech-to-text
+    - audio-transcription
+    - vllm
+    - gpu
+    - mistral
+  overrides:
+    backend: vllm
+    known_usecases:
+      - transcript
+    parameters:
+      model: mistralai/Voxtral-Mini-4B-Realtime-2602
+    options:
+      - trust_remote_code:true
+
 - name: voxtral-mini-4b-realtime
   license: apache-2.0
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"

gallery/voxtral-mini-4b-realtime-vllm.yaml

@@ -0,0 +1,36 @@
+---
+name: "voxtral-mini-4b-realtime-vllm"
+
+config_file: |
+    backend: vllm
+    context_size: 8192
+    parameters:
+      max_tokens: 8192
+    function:
+      disable_no_action: true
+      grammar:
+        disable: true
+        parallel_calls: true
+        expect_strings_after_json: true
+    template:
+      use_tokenizer_template: true
+    # Uncomment to specify a quantization method (optional)
+    # quantization: "awq"
+    # Uncomment to set dtype, choices are: "auto", "half", "float16", "bfloat16", "float", "float32". awq on vLLM does not support bfloat16
+    # dtype: "float16"
+    # Uncomment to limit the GPU memory utilization (vLLM default is 0.9 for 90%)
+    # gpu_memory_utilization: 0.5
+    # Uncomment to trust remote code from huggingface
+    trust_remote_code: true
+    # Uncomment to enable eager execution
+    # enforce_eager: true
+    # Uncomment to specify the size of the CPU swap space per GPU (in GiB)
+    # swap_space: 2
+    # Uncomment to specify the maximum length of a sequence (including prompt and output)
+    # max_model_len: 32768
+    # Uncomment and specify the number of Tensor divisions.
+    # Allows you to partition and run large models. Performance gains are limited.
+    # https://github.com/vllm-project/vllm/issues/1435
+    # tensor_parallel_size: 2
+    # Uncomment to disable log stats
+    # disable_log_stats: true