Skip to content

Commit bd3d363

Browse files
authored
ci: raise zb bench HTTP timeouts and wait for zot shutdown (#4219)
zb GC stress and benchmark jobs were flaking when slow chunked pushes or large pulls approached zot's 60s default read/write timeout. Set 2m timeouts on all zb bench configs and use killall --wait so zot can shut down cleanly before CI collects logs. See: https://github.com/project-zot/zot/actions/runs/29038448071/job/86317480369?pr=4210 ``` Test name: Push Chunk Mixed 33% 1MB, 33% 10MB, 33% 100MB Time taken for tests: 1m0.281970856s Requests per second: 1.6588708 Complete requests: 99 Failed requests: 1 Error Patch "http://localhost:8080/v2/a960d36a-7c37-11f1-92d0-02001713c410/blobs/uploads/a6772496-8c5e-46d2-b1d6-01fb47440b66": readfrom tcp 127.0.0.1:53292->127.0.0.1:8080: write tcp 127.0.0.1:53292->127.0.0.1:8080: use of closed network connection count: 1 ``` Signed-off-by: Andrei Aaron <andreifdaaron@gmail.com>
1 parent 8759839 commit bd3d363

11 files changed

Lines changed: 33 additions & 15 deletions

.github/workflows/gc-stress-test.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ jobs:
3434
sleep 10
3535
bin/zb-linux-amd64 -c 10 -n 100 -o ci-cd http://localhost:8080
3636
37-
killall -r zot-*
37+
killall --wait -r zot-*
3838
3939
# clean zot storage
4040
sudo rm -rf /tmp/zot
@@ -76,7 +76,7 @@ jobs:
7676
sleep 10
7777
bin/zb-linux-amd64 -c 10 -n 100 -o ci-cd http://localhost:8080
7878
79-
killall -r zot-*
79+
killall --wait -r zot-*
8080
8181
# clean zot storage
8282
sudo rm -rf /tmp/zot
@@ -158,7 +158,7 @@ jobs:
158158
sleep 10
159159
bin/zb-linux-amd64 -c 10 -n 100 -o ci-cd http://localhost:8080
160160
161-
killall -r zot-*
161+
killall --wait -r zot-*
162162
163163
# clean zot storage
164164
sudo rm -rf /tmp/zot
@@ -245,7 +245,7 @@ jobs:
245245
sleep 10
246246
bin/zb-linux-amd64 -c 10 -n 100 -o ci-cd http://localhost:8080
247247
248-
killall -r zot-*
248+
killall --wait -r zot-*
249249
250250
# clean zot storage
251251
sudo rm -rf /tmp/zot

.github/workflows/nightly.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ jobs:
100100
sleep 10
101101
bin/zb-linux-amd64 -c 10 -n 100 -o ci-cd http://localhost:8080 --skip-cleanup
102102
103-
killall -r zot-*
103+
killall --wait -r zot-*
104104
105105
# clean zot storage
106106
sudo rm -rf /tmp/zot
@@ -146,7 +146,7 @@ jobs:
146146
sleep 10
147147
bin/zb-linux-amd64 -c 10 -n 100 -o ci-cd http://localhost:8080 --skip-cleanup
148148
149-
killall -r zot-*
149+
killall --wait -r zot-*
150150
151151
# clean zot storage
152152
sudo rm -rf /tmp/zot

examples/config-bench.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@
55
},
66
"http": {
77
"address": "127.0.0.1",
8-
"port": "8080"
8+
"port": "8080",
9+
"readTimeout": "2m",
10+
"writeTimeout": "2m"
911
},
1012
"log": {
1113
"level": "debug",

test/cluster/config-minio-redis.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,9 @@
2020
},
2121
"http": {
2222
"address": "127.0.0.1",
23-
"port": "8081"
23+
"port": "8081",
24+
"readTimeout": "2m",
25+
"writeTimeout": "2m"
2426
},
2527
"log": {
2628
"level": "debug",

test/cluster/config-minio.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,9 @@
1616
},
1717
"http": {
1818
"address": "127.0.0.1",
19-
"port": "8081"
19+
"port": "8081",
20+
"readTimeout": "2m",
21+
"writeTimeout": "2m"
2022
},
2123
"log": {
2224
"level": "debug",

test/gc-stress/config-gc-bench-local.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@
88
},
99
"http": {
1010
"address": "127.0.0.1",
11-
"port": "8080"
11+
"port": "8080",
12+
"readTimeout": "2m",
13+
"writeTimeout": "2m"
1214
},
1315
"log": {
1416
"level": "debug",

test/gc-stress/config-gc-bench-s3-localstack.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,9 @@
2929
},
3030
"http": {
3131
"address": "127.0.0.1",
32-
"port": "8080"
32+
"port": "8080",
33+
"readTimeout": "2m",
34+
"writeTimeout": "2m"
3335
},
3436
"log": {
3537
"level": "debug",

test/gc-stress/config-gc-bench-s3-minio.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,9 @@
3131
},
3232
"http": {
3333
"address": "127.0.0.1",
34-
"port": "8080"
34+
"port": "8080",
35+
"readTimeout": "2m",
36+
"writeTimeout": "2m"
3537
},
3638
"log": {
3739
"level": "debug",

test/gc-stress/config-gc-referrers-bench-local.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,9 @@
1717
},
1818
"http": {
1919
"address": "127.0.0.1",
20-
"port": "8080"
20+
"port": "8080",
21+
"readTimeout": "2m",
22+
"writeTimeout": "2m"
2123
},
2224
"log": {
2325
"level": "debug",

test/gc-stress/config-gc-referrers-bench-s3-localstack.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,9 @@
3838
},
3939
"http": {
4040
"address": "127.0.0.1",
41-
"port": "8080"
41+
"port": "8080",
42+
"readTimeout": "2m",
43+
"writeTimeout": "2m"
4244
},
4345
"log": {
4446
"level": "debug",

0 commit comments

Comments
 (0)