You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+11-11Lines changed: 11 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ A Viam provided ML Model service backed by NVIDIA's [Triton Inference Server](ht
4
4
Configure this ML Model service as a modular resource on your machine with a [Jetson board](https://docs.viam.com/build/configure/components/board/jetson/) or another supported machine to deploy [supported models](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/backend/docs/backend_platform_support_matrix.html) to your machine faster while consuming less computer power.
5
5
6
6
> [!Note]
7
-
> For more information, see the [ML Model service documentation](/ml/deploy/).
7
+
> For more information, see the [ML Model service documentation](https://docs.viam.com/dev/reference/apis/services/ml/).
8
8
9
9
## Requirements
10
10
@@ -13,7 +13,7 @@ An NVIDIA Jetson Orin board or other machine with an NVIDIA GPU with the followi
13
13
1. The NVIDIA Container Runtime (which can be installed with `sudo apt-get install nvidia-container`)
14
14
2. On machines that support it, [Jetpack 5 or Jetpack 6](https://developer.nvidia.com/embedded/jetpack) (which can be installed with `sudo apt-get install nvidia-jetpack`)
15
15
16
-
Then, if you haven't done so already, create a new robot in [the Viam app](https://app.viam.com).
16
+
Then, if you haven't done so already, create a new robot in [Viam](https://app.viam.com).
17
17
[Install `viam-server` on the board](https://docs.viam.com/get-started/installation/prepare/jetson-agx-orin-setup/) and connect to the robot.
18
18
19
19
## Install from Viam registry
@@ -35,9 +35,9 @@ directory of this repo.
35
35
## Configure your Triton ML Model Service
36
36
37
37
> [!Note]
38
-
> Before configuring your ML Model service module, you must [create a robot](https://docs.viam.com/manage/fleet/robots/#add-a-new-robot).
38
+
> Before configuring your ML Model service module, you must [create a machine](https://docs.viam.com/operate/get-started/setup/).
39
39
40
-
Navigate to the **Config** tab of your robot’s page in [the Viam app](https://app.viam.com/).
40
+
Navigate to the **Config** tab of your machine's page in [Viam](https://app.viam.com/).
41
41
Click on the **Services** subtab and click **Create service**.
42
42
Select **ML Model**, then search for the `mlmodelservice:triton` model.
43
43
Click **Add module**.
@@ -58,17 +58,17 @@ Your `"modules"` array should have an entry like the following:
58
58
(The version used might differ from this example.)
59
59
Save your config.
60
60
61
-
> [!NOTE]
62
-
> For more information, see [Configure a Robot](https://docs.viam.com/manage/configuration/).
63
-
64
61
Now, to configure the service's **Attributes**, you have two options.
65
62
You can use a TensorFlow or PyTorch model from the [Registry](https://app.viam.com/registry), or you can load an existing TensorFlow or PyTorch model on your machine.
66
63
To deploy a model from the Registry, navigate back to the **Config** tab of your machine in the Viam app and switch back to **Builder** mode.
67
64
Click on the **Select ML model** field and select a model from the dropdown that appears.
68
-
Your ML model service will automatically be configured with this model.
65
+
Your ML model service will automatically reconfigure to use this model.
69
66
You can explore the available models in the [Registry](https://app.viam.com/registry).
70
67
71
-
To deploy an existing model on your machine, you can either 1) specify the name of a model downloaded from the Viam registry, or 2) [create your own local model repository](#create-a-repository-to-store-the-ml-model-to-deploy) on your machine.
68
+
To deploy an existing model on your machine, you can:
69
+
70
+
- specify the name of a model downloaded from the Viam registry
71
+
-[create your own local model repository](#create-a-repository-to-store-the-ml-model-to-deploy) on your machine.
72
72
73
73
### Option 1: use a model from the Viam registry
74
74
@@ -154,7 +154,7 @@ The following attributes are available for the MLModel service `viam:mlmodelserv
154
154
|`backend_directory`| string | Optional | A container side path to the TritonServer "backend" directory. You normally do not need to override this; the build will set it to the backend directory of the Triton Server installation in the container. You may set it if you wish to use a different set of backends. |
155
155
|`model_version`| int | Optional | The version of the model to be loaded from `model_repository_path`. If not specified, the module will use the newest version of the model named by model_name.<br><br>Default: `-1` (newest) |
156
156
|`tensor_name_remappings`| obj | Optional | Provides two dictionaries under the `inputs` and `outputs` keys that rename the models' tensors. Other Viam services, like the [vision service]([/ml/vision/](https://docs.viam.com/registry/advanced/mlmodel-design/)) may expect to work with tensors with particular names. Use this map to rename the tensors from the loaded model to what the vision service expects as needed to meet those requirements.<br><br>Default: `{}`|
157
-
|`model_config`| obj | Optional |[Triton Model Configuration Parameters](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/model_configuration.html). If provided, this setting will override any configuration in the configured repository. |
157
+
|`model_config`| obj | Optional |Supports all [Triton Model Configuration Parameters](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/model_configuration.html). If provided, this setting overrides configuration in the model repository. Use this attribute to configure performance tweaks such as [instance groups](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/model_configuration.html#instance-groups) (multiple instances of a model) and [dynamic batching](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/optimization.html#dynamic-batcher) (combining requests into batches). |
158
158
159
159
### Example configurations
160
160
@@ -232,4 +232,4 @@ If you have your own Triton model repository, you could use it like this:
232
232
}
233
233
```
234
234
235
-
You can now connect this vision service to a [transform camera](https://docs.viam.com/build/configure/components/camera/transform/), or get detections programmatically through one of Viam's [client SDKs](https://docs.viam.com/build/program/).
235
+
You can now connect this vision service to a [transform camera](https://docs.viam.com/build/configure/components/camera/transform/), or get detections programmatically through one of Viam's [client SDKs](https://docs.viam.com/build/program/).
0 commit comments