File tree Expand file tree Collapse file tree
inference/qwen3/conf/inference Expand file tree Collapse file tree Original file line number Diff line number Diff line change 1919 tensor_parallel_size : 2
2020 pipeline_parallel_size : 1
2121 gpu_memory_utilization : 0.5
22+ # KS3-backed checkpoints must be read sequentially. Lazy mmap loading can
23+ # trigger slow random reads and native worker crashes on MUSA.
24+ safetensors_load_strategy : eager
2225 max_model_len : 1024
2326 seed : 1234
2427 enforce_eager : true
Original file line number Diff line number Diff line change @@ -31,6 +31,7 @@ experiment:
3131 FS_PLATFORM : musa
3232 TORCHDYNAMO_DISABLE : 1
3333 TORCH_COMPILE_DISABLE : 1
34+ USE_FLAGGEMS : " false"
3435 VLLM_WORKER_MULTIPROC_METHOD : spawn
3536 MUSA_VISIBLE_DEVICES : " 0"
3637 MTHREADS_VISIBLE_DEVICES : all
Original file line number Diff line number Diff line change 1919 tensor_parallel_size : 1
2020 pipeline_parallel_size : 1
2121 gpu_memory_utilization : 0.5
22+ # Avoid lazy mmap reads from the network-backed model volume.
23+ safetensors_load_strategy : eager
2224 max_model_len : 1024
2325 max_num_seqs : 16
2426 enforce_eager : true
You can’t perform that action at this time.
0 commit comments