update README and update fp8 gemm dispatch for gfx95

[zhuyuhua-v]

Purpose

1. update evaluation README.md
2. update fp8 gemm dispatch for gfx95 [ROCm][GEMM] update aiter fp8 linear check for different platforms vllm-project/vllm#27579
   This PR targets ROCm FP8 gemm, since AITER’s FP8 GEMM implementation now supports multiple platforms, the previous is_fp8_fnuz check is no longer necessary.
   Test results show that after removing this is_fp8_fnuz check, AITER FP8 GEMM consistently outperforms the original VLLM Triton GEMM on MI355 across various scenarios, while maintaining accuracy.

Test Plan

The following tests validate DeepSeek-V3 model by collecting benchmark metrics and performing correctness tests through lm_eval.

vLLM server launch command:

VLLM_USE_V1=1 \
SAFETENSORS_FAST_GPU=1 \
VLLM_ROCM_USE_AITER=1 \
VLLM_ROCM_USE_AITER_MOE=1 \
VLLM_USE_TRITON_FLASH_ATTN=0 \
NCCL_DEBUG=WARN \
VLLM_RPC_TIMEOUT=1800000 \
VLLM_ROCM_USE_AITER_MHA=0 \
VLLM_ROCM_USE_TRITON_ROPE=1 \
VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 \

vllm serve ${model_path} --tensor-parallel-size 8 --max-num-batched-tokens 32768 --trust-remote-code --no-enable-prefix-caching --disable-log-requests --compilation-config '{"cudagraph_mode": "FULL_AND_PIECEWISE"}' --gpu_memory_utilization 0.9 --block-size 1

Benchmark commands:

vllm bench serve --host localhost --port 8000 --model ${model} --dataset-name random --random-input-len ${input_token_len} --random-output-len ${output_token_len}  --max-concurrency ${max_concurrency} --num-prompts ${num_prompts} --percentile-metrics ttft,tpot,itl,e2el --ignore-eos --seed 123

Accuracy commands:

lm_eval --model local-completions --tasks gsm8k --model_args model=${model},base_url=http://127.0.0.1:8000/v1/completions --batch_size 100 

Test Result

Benchmark Result

deepseek-ai/DeepSeek-V3 on MI355, input-len/output-len: 1k/1k

aiter fp 8 gemm	max_concurrency	QPS	TTFT		TPOT		ITL
			Mean	P99	Mean	P99	Mean	P99
Yes	4	0.19	165.52	212.49	20.52	20.59	20.52	20.96
No	4	0.17	184.66	221.76	23.3	23.41	23.3	23.78
+		11.76%	11.56%	4.36%	13.55%	13.70%	13.55%	13.45%
Yes	32	1.29	1062.03	1267.41	23.18	24.09	23.18	23.61
No	32	1.14	859.43	1128.97	26.52	27.22	26.52	27.14
+		13.16%	-19.08%	-10.92%	14.41%	12.99%	14.41%	14.95%
Yes	64	2.2	1645.79	2501.27	26.77	28.36	26.77	26.9
No	64	2	1491.23	2238.82	29.76	31.11	29.76	30.21
+		10.00%	-9.39%	-10.49%	11.17%	9.70%	11.17%	12.30%
Yes	128	3.58	2791.65	4838.39	32.23	34.8	32.23	32.64
No	128	3.31	2293.06	4182.16	35.55	37.85	35.55	35.77
+		8.16%	-17.86%	-13.56%	10.30%	8.76%	10.30%	9.59%

deepseek-ai/DeepSeek-V3 on MI355, input-len/output-len: 8k/1k

aiter fp 8 gemm	max_concurrency	QPS	TTFT		TPOT		ITL
			Mean	P99	Mean	P99	Mean	P99
Yes	4	0.17	1125.09	1396.97	21.48	22.29	21.48	21.77
No	4	0.15	1048.77	1288.84	24.26	24.94	24.26	24.48
+		13.33%	-6.78%	-7.74%	12.94%	11.89%	12.94%	12.45%
Yes	32	0.85	5636.24	11484.02	31.23	36.3	31.23	36.99
No	32	0.8	5075.97	10372.23	34.02	38.55	34.02	37.37
+		6.25%	-9.94%	-9.68%	8.93%	6.20%	8.93%	1.03%
Yes	64	1.17	7993.25	22929.74	45.56	52.73	45.56	1039.87
No	64	1.15	6550.93	20664.27	48.04	53.87	48.04	998.8
+		1.74%	-18.04%	-9.88%	5.44%	2.16%	5.44%	-3.95%
Yes	128	1.47	10446.57	44693.02	74.62	83.58	74.62	1418.05
No	128	1.5	9360.78	40097.91	74.33	82.34	74.33	1368.9
+		-2.00%	-10.39%	-10.28%	-0.39%	-1.48%	-0.39%	-3.47%

Acuracy Result

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.9515	±	0.0059
		strict-match	5	exact_match	↑	0.9515	±	0.0058