Skip to content

Commit c0a35c0

Browse files
authored
fix-broken-link (#729)
1 parent 93147e5 commit c0a35c0

1 file changed

Lines changed: 6 additions & 10 deletions

File tree

docs/tutorials/demo.md

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# vllm with wva autoscaler
22

33

4-
Notes:
4+
Notes:
55
1. Experiments on OpenShift Cluster with H100 GPUs.
66
2. To setup `vLLM` on `Openshift`, refer to [vllm-samples.md](vllm-samples.md).
7-
3. We use `guidellm` as the load generator. Refer to [guidellm-sample.md](guidellm-sample.md) for a quick tutorial to create your guidellm image that will be used in a `Job` resource.
7+
3. We use `guidellm` as the load generator. Refer to [guidellm-sample.md](guidellm-sample.md) for a quick tutorial to create your guidellm image that will be used in a `Job` resource.
88
3. The WVA autoscaler is assumed to be deployed in `workload-variant-autoscaler-system` namespace.
99

1010

@@ -96,19 +96,19 @@ spec:
9696
- "--data"
9797
- "prompt_tokens=128,output_tokens=512"
9898
- "--output-path"
99-
- "/tmp/benchmarks.json"
99+
- "/tmp/benchmarks.json"
100100
restartPolicy: Never
101101
backoffLimit: 4
102102
```
103103

104104
In each job, fill in `image: <image-repo>:<tag>` with your `guidellm` image repo and tag. The `<rate>` and `max-seconds` are set as follows.
105105

106-
- In `guidellm-job-1.yaml`, we set `<rate>` and `<max-seconds>` to `8` and `1800` respectively. By doing this, we force `guidellm` client to send requests at rate `8` requests per second (480 req/min) for `30` minutes.
106+
- In `guidellm-job-1.yaml`, we set `<rate>` and `<max-seconds>` to `8` and `1800` respectively. By doing this, we force `guidellm` client to send requests at rate `8` requests per second (480 req/min) for `30` minutes.
107107
- In `guidellm-job-2.yaml`, we set `<rate>` and `<max-seconds>` to `8` and `1200` respectively. We start this job after a couple of minutes of starting `guidellm-job-1`. When both jobs are running, we are effectively sending requests at rate `8+8 = 16` requests per second (960 req/min).
108108
- In `guidellm-job-3.yaml`, we set `<rate>` and `<max-seconds>` to `8` and `720` respectively. We start this job after a couple of minutes of starting `guidellm-job-2`. When all the three jobs are running, we are effectively sending requests at rate `8+8+8 = 24` requests per second (1440 req/min) for 12 minutes.
109109
- With this setup, `guidellm-job-3` will complete first, bringing the effective request rate back to `16` req/sec. This is followed by the completion of `guidellm-job-2`, which will bring down rate to `8` req/sec. Finally, `guidellm-job-1` completes, after which no further requests are sent.
110110

111-
**Dynamic Load Generation Summary:**
111+
**Dynamic Load Generation Summary:**
112112
- Step 1: `oc apply -f guidellm-job-1.yaml`. Wait about 5 minutes before continuing to step 2.
113113
- Step 2: `oc apply -f guidellm-job-2.yaml`. Wait about 5 minutes before continuing to step 3.
114114
- Step 3: `oc apply -f guidellm-job-3.yaml`
@@ -118,8 +118,4 @@ In each job, fill in `image: <image-repo>:<tag>` with your `guidellm` image repo
118118
## WVA Performance
119119
The following figure shows the behaviour observed from the controller logs.
120120

121-
![Autoscaler Diagram](../../docs/diagrams/autoscaler-demo.png)
122-
123-
124-
125-
121+
![Autoscaler Diagram](../design/diagrams/autoscaler-demo.png)

0 commit comments

Comments
 (0)