Skip to content

Commit f2ebb4b

Browse files
committed
Add deepseek-r1 32b back
1 parent f296c12 commit f2ebb4b

2 files changed

Lines changed: 17 additions & 10 deletions

File tree

docs/SETUP.md

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1106,13 +1106,15 @@ Once you're done with all of the above, you'll need to go into your BIOS and ens
11061106

11071107
#### GPU Node Setup
11081108

1109+
> God help you. Good luck.
1110+
11091111
Follow Nvidia's setup steps here for `nvidia-container-toolkit`:
11101112

11111113
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
11121114

1115+
Also, follow the prerequisites setup steps from Nvidia for their container toolkit here:
11131116

1114-
1115-
TODO
1117+
https://github.com/NVIDIA/k8s-device-plugin?tab=readme-ov-file#install-the-nvidia-container-toolkit
11161118

11171119
Test with `containerd` that the Nvidia GPU is available:
11181120

@@ -1124,13 +1126,7 @@ sudo ctr run --rm --gpus 0 -t docker.io/nvidia/cuda:12.3.2-base-ubuntu22.04 cuda
11241126

11251127
If there are any issues with `containerd`, make sure that the following is set up for the `/etc/containerd/config.d/99-nvidia.toml` file:
11261128

1127-
```toml
1128-
1129-
```
1130-
1131-
#### Helpful Commands
1132-
1133-
````bash
1129+
```bash
11341130
version = 2
11351131

11361132
[plugins]
@@ -1151,4 +1147,10 @@ version = 2
11511147

11521148
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.nvidia.options]
11531149
BinaryName = "/usr/bin/nvidia-container-runtime"
1154-
````
1150+
```
1151+
1152+
TODO more explanation on setup
1153+
1154+
#### Helpful Commands
1155+
1156+
TODO

helm/ollama/values.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,18 @@ ollama:
1616
nvidiaResource: "nvidia.com/gpu"
1717
models:
1818
pull:
19+
- deepseek-r1:32b
1920
- gpt-oss:20b
2021
- qwen2.5vl:7b
2122
create:
2223
- name: gpt-oss:20b-64k-context
2324
template: |
2425
FROM gpt-oss:20b
2526
PARAMETER num_ctx 65536
27+
- name: deepseek-r1:32b-64k-context
28+
template: |
29+
FROM deepseek-r1:32b
30+
PARAMETER num_ctx 65536
2631
insecure: false
2732
mountPath: ""
2833

0 commit comments

Comments
 (0)