Merge upstream→gfx11 by eble-amd · Pull Request #1014 · ROCm/vllm

eble-amd · 2026-06-22T15:58:52Z

Merge from upstream. Most conflicts were automatically resolved by a three-way merge tool. There were just a few instances of both sides adding code in the same place; I kept it all.

Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…ct#44338) Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

…mask (vllm-project#44082) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude <noreply@anthropic.com>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

…llm-project#42027) Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com> Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>

Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>

Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

… supports check (vllm-project#43332) Signed-off-by: Junhao Shen <junshen@nvidia.com> Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>

…t references it (vllm-project#44128) Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

…on Top-p path. (vllm-project#42191) Signed-off-by: js_park <cakeng@naver.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>

…44356) Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>

…oject#44366) Signed-off-by: Kevin H. Luu <khluu000@gmail.com>

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

…t#44368) Signed-off-by: JartX <sagformas@epdcenter.es>

Signed-off-by: JartX <sagformas@epdcenter.es>

…llm-project#44042) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…project#44352) Signed-off-by: sfeng33 <4florafeng@gmail.com>

Signed-off-by: Rotem Shavitt <rshavitt@gmail.com>

…oject#44236) Signed-off-by: Willow Lopez <100782273+Oxygen56@users.noreply.github.com>

) Signed-off-by: William-Rom <william.rom@intility.no> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

…t-template-kwargs` for client-rendered datasets (vllm-project#44244) Signed-off-by: Albert Cheng <albertching0112@gmail.com>

…ct#42758) Signed-off-by: charlifu <charlifu@amd.com>

…-bake script (vllm-project#36949) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

vllm-project#44251) Signed-off-by: Majid Taheri Andani <tahemaji@amazon.com> Co-authored-by: Majid Taheri Andani <tahemaji@amazon.com> Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>

…t#44287) Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com> Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>

Signed-off-by: NolanHo <kujyo.eia.serias@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

…roject#44311) Signed-off-by: Bugen Zhao <i@bugenzhao.com>

…ect#44771) Signed-off-by: Yan Ma <yan.ma@intel.com>

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

… rare OOMs (vllm-project#44761) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

Signed-off-by: zengxian <xiangdong.zeng@intel.com>

…llm-project#44828) Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com> Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>

…ct#44499) Signed-off-by: Sahil Singh <sahiilsiingh37@gmail.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

…rence failures (vllm-project#44470) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>

…eloaded (vllm-project#44419) Signed-off-by: jmamou <jonathan.mamou@intel.com> Signed-off-by: Jonathan Mamou <jonathan.mamou@intel.com> Co-authored-by: Li, Jiang <bigpyj64@gmail.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com>

Signed-off-by: NickLucche <nlucches@redhat.com>

…ngs (vllm-project#44852) Signed-off-by: jiang1.li <jiang1.li@intel.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

…for random dataset (vllm-project#44708)

Signed-off-by: walterbm <walter.beller.morales@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

mgehre-amd · 2026-06-22T19:48:53Z

Looks good! Did you do some smoke test by running some model before/after the change?

eble-amd · 2026-06-22T19:51:22Z

Looks good! Did you do some smoke test by running some model before/after the change?

No, I thought our github workflows were testing.

eble-amd · 2026-06-22T19:54:21Z

Hm... I didn't notice till now that there are conflicts preventing merging. I'll convert this back to a draft to investigate.

Merge from upstream. Most conflicts were automatically resolved by a three-way merge tool. There were just a few instances of both sides adding code in the same place; I kept it all. Signed-off-by: Dan Eble <Dan.Eble@amd.com>

eble-amd · 2026-06-23T18:23:37Z

Benchmark of model Qwen3-4B_128:

this PR on my assigned test system:

Prefill 2566.35 tokens/s; TTFT 50 ms
Decode 73.6 tokens/s (TPOT 13.59 ms)
End-to-end latency 1776 ms (median)
Average CPU utilization 6%

nightly regression suite:

Prefill 2410.4 tokens/s; TTFT 53.1 ms
Decode 73.4 tokens/s (TPOT 13.6 ms)
End-to-end latency 1,783 ms (median)
Average CPU utilization 6%

mgehre-amd · 2026-06-23T21:08:46Z

I merged @marcusr-amd first and this seems to have caused conflicts here. Can you please resolve them?

dependabot Bot and others added 30 commits June 2, 2026 11:26

Bump actions/github-script from 8.0.0 to 9.0.0 (vllm-project#39667)

b8b49e2

Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[MRV2] Remove assignment of graph_pool in cudagraph_utils (vllm-proje…

e4a2e58

…ct#44338) Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[Bugfix] Cache the EAGLE/MTP lookahead block in the SWA prefix-cache …

e9e08c4

…mask (vllm-project#44082) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>

[Misc] Remove stray empty file (vllm-project#44350)

5577811

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude <noreply@anthropic.com>

[ModelRunnerV2] Avoid pipeline parallel bubbles (vllm-project#42187)

e15f202

Signed-off-by: Nick Hill <nickhill123@gmail.com>

[Kernel][MoE] Add GELU_TANH to CPU, CUTLASS, and WNA16 MoE backends (v…

3099de3

…llm-project#42027) Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com> Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>

Fix sparse NCCL weight transfer test construction (vllm-project#44345)

0917a00

Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>

[CI/Build] Bump flashinfer to v0.6.12 (vllm-project#44036)

8b3b71e

Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

[MoE/b12x] Accept W4A16 (kNvfp4Static, None) in FlashInferB12xExperts…

a4ac746

… supports check (vllm-project#43332) Signed-off-by: Junhao Shen <junshen@nvidia.com> Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>

[Misc] Remove dead VLLM_RPC_TIMEOUT env var and fix profiling doc tha…

bd98e97

…t references it (vllm-project#44128) Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>

[DSV4] Minor cleanup for DeepseekV4MegaMoEExperts (vllm-project#44367)

b254e04

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[Perf] Apply single-pass min_larger finding and binary search in Trit…

ca17b6b

…on Top-p path. (vllm-project#42191) Signed-off-by: js_park <cakeng@naver.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>

[Bugfix] Fix Deepseek v4 non-mega-moe model init error (vllm-project#…

969aec4

…44356) Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>

[docker] Stop using extra-index-url for flashinfer-jit-cache (vllm-pr…

27a93cd

…oject#44366) Signed-off-by: Kevin H. Luu <khluu000@gmail.com>

[Platform] Add is_cumem_allocator_available (vllm-project#43838)

02a0149

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

[ROCm][CI] Fix stale wvSplitK GEMM fallback test for N=5 (vllm-projec…

4454a18

…t#44368) Signed-off-by: JartX <sagformas@epdcenter.es>

[ROCm][CI] Skip fp8 reload tests on gfx90a (MI250) (vllm-project#44369)

7b476c8

Signed-off-by: JartX <sagformas@epdcenter.es>

[CI] Reject out-of-vocabulary before they reach the GPU logprob path (v…

53b88d1

…llm-project#44042) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[CI] Add missing vllm/parser/ CI trigger and fix test_parse.py (vllm-…

e670638

…project#44352) Signed-off-by: sfeng33 <4florafeng@gmail.com>

Nit Changes in Tiered KV Offload (vllm-project#44293)

3f0a91b

Signed-off-by: Rotem Shavitt <rshavitt@gmail.com>

fix: resolve CUTLASS fmin compatibility for DeepSeek-V4 init (vllm-pr…

597bc15

…oject#44236) Signed-off-by: Willow Lopez <100782273+Oxygen56@users.noreply.github.com>

[Bugfix] fix crash in postprocess for null tool args (vllm-project#43862

f020435

) Signed-off-by: William-Rom <william.rom@intility.no> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

[Benchmark] Enable reasoning-model (thinking) benchmarking via `--cha…

e0081ef

…t-template-kwargs` for client-rendered datasets (vllm-project#44244) Signed-off-by: Albert Cheng <albertching0112@gmail.com>

Enable perf_token_group_quant/_C_stable_libtorch for ROCm (vllm-proje…

71df063

…ct#42758) Signed-off-by: charlifu <charlifu@amd.com>

[ROCm][CI] Optimize ROCm Docker build: registry cache, DeepEP, and ci…

87954eb

…-bake script (vllm-project#36949) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Perf] Add tuned selective_state_update configs for H200 and RTX PRO … (

9af53a3

vllm-project#44251) Signed-off-by: Majid Taheri Andani <tahemaji@amazon.com> Co-authored-by: Majid Taheri Andani <tahemaji@amazon.com> Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>

[KV Offloading] Enable HMA models for Tiering Offloading (vllm-projec…

7268457

…t#44287) Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com> Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>

[Rust Frontend] Add server router extension hook (vllm-project#43774)

4aaed4c

Signed-off-by: NolanHo <kujyo.eia.serias@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

[Rust Frontend] Add dynamic LoRA endpoints (vllm-project#43778)

6550ff1

Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

[Rust Frontend] Fix several hf chat template rendering issues (vllm-p…

449be4f

…roject#44311) Signed-off-by: Bugen Zhao <i@bugenzhao.com>

yma11 and others added 13 commits June 8, 2026 13:58

[XPU][Minor] format moe kernel name and add in kernel list (vllm-proj…

54c660c

…ect#44771) Signed-off-by: Yan Ma <yan.ma@intel.com>

[ROCm][CI] Stage C mirrors (vllm-project#42793)

967c5c3

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[ROCm][CI] Stabilizing teardown and timeout of flaky tests to prevent…

d9ff7e4

… rare OOMs (vllm-project#44761) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[XPU][CI] Add more test cases in Intel GPU CI (vllm-project#43663)

94fcdd0

Signed-off-by: zengxian <xiangdong.zeng@intel.com>

[BugFix] Use served model name in gemma4 audio-tower error message (v…

469f3dc

…llm-project#44828) Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com> Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>

[Rust Frontend] Add /pause, /resume, /is_paused endpoints (vllm-proje…

3c0b443

…ct#44499) Signed-off-by: Sahil Singh <sahiilsiingh37@gmail.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

[XPU] Cap topk/topp Triton BLOCK_SIZE to 4096 to fix Top-p mask diffe…

fa662b1

…rence failures (vllm-project#44470) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>

[Connector] Remove P2pNcclConnector (vllm-project#44854)

5add018

Signed-off-by: NickLucche <nlucches@redhat.com>

[CI/Build][CPU] Fix flaky CI image build failure and unexpected warni…

980796c

…ngs (vllm-project#44852) Signed-off-by: jiang1.li <jiang1.li@intel.com>

[CI] Consolidate multimodal entrypoint tests. (vllm-project#44819)

93ee4cd

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

[Benchmark] Auto-detect and correct client/server tokenizer mismatch …

ac3409d

…for random dataset (vllm-project#44708)

[Quantization] add online fp8 ptpc (vllm-project#44132)

753e9d5

Signed-off-by: walterbm <walter.beller.morales@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

eble-amd force-pushed the merge-from-upstream branch from fc7be7f to 8be8c01 Compare June 22, 2026 17:13

eble-amd requested a review from mgehre-amd June 22, 2026 18:50

eble-amd marked this pull request as ready for review June 22, 2026 18:50

eble-amd requested review from AndreasKaratzas and dllehr-amd as code owners June 22, 2026 18:50

mgehre-amd approved these changes Jun 22, 2026

View reviewed changes

mgehre-amd removed request for AndreasKaratzas and dllehr-amd June 22, 2026 19:48

eble-amd marked this pull request as draft June 22, 2026 19:54

Merge commit '753e9d55e6' into merge-from-upstream

a3e798b

Merge from upstream. Most conflicts were automatically resolved by a three-way merge tool. There were just a few instances of both sides adding code in the same place; I kept it all. Signed-off-by: Dan Eble <Dan.Eble@amd.com>

eble-amd force-pushed the merge-from-upstream branch from 8be8c01 to a3e798b Compare June 22, 2026 20:24

eble-amd marked this pull request as ready for review June 23, 2026 18:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge upstream→gfx11#1014

Merge upstream→gfx11#1014
eble-amd wants to merge 183 commits into
ROCm:gfx11from
eble-amd:merge-from-upstream

eble-amd commented Jun 22, 2026 •

edited

Loading

Uh oh!

mgehre-amd commented Jun 22, 2026

Uh oh!

eble-amd commented Jun 22, 2026

Uh oh!

eble-amd commented Jun 22, 2026

Uh oh!

eble-amd commented Jun 23, 2026

Uh oh!

mgehre-amd commented Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

eble-amd commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mgehre-amd commented Jun 22, 2026

Uh oh!

eble-amd commented Jun 22, 2026

Uh oh!

eble-amd commented Jun 22, 2026

Uh oh!

eble-amd commented Jun 23, 2026

Uh oh!

mgehre-amd commented Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

eble-amd commented Jun 22, 2026 •

edited

Loading