Skip to content

Commit a345605

Browse files
committed
update README and update fp8 gemm dispatch for gfx95
Signed-off-by: zhuyuhua-v <yuhzhu@amd.com>
1 parent f6f462b commit a345605

2 files changed

Lines changed: 4 additions & 4 deletions

File tree

evaluation/README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,8 @@
108108
--disable-log-requests \
109109
--compilation-config '{"cudagraph_mode": "FULL_AND_PIECEWISE"}' \
110110
--gpu_memory_utilization 0.9 \
111-
--block-size 1
111+
--block-size 1 \
112+
--async-scheduling
112113
```
113114

114115
## Curl request

vllm/model_executor/layers/quantization/utils/fp8_utils.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -945,13 +945,12 @@ def requant_weight_ue8m0_inplace(
945945

946946

947947
def check_aiter_fp8_linear_support() -> bool:
948-
"""AITER is only supported on ROCm and only for FP8_FNUZ
949-
and at the moment are MI300 series"""
948+
"""AITER is only supported on ROCm"""
950949
return (
951950
current_platform.is_rocm()
952951
and envs.VLLM_ROCM_USE_AITER
953952
and envs.VLLM_ROCM_USE_AITER_LINEAR
954-
and current_platform.is_fp8_fnuz()
953+
and current_platform.supports_fp8()
955954
)
956955

957956

0 commit comments

Comments
 (0)