Fa upstream3 by fsx950223 · Pull Request #560 · ROCm/vllm

fsx950223 · 2025-05-29T06:29:24Z

Please direct your PRs to the upstream vllm (https://github.com/vllm-project/vllm.git)

Accepting PRs into the ROCm fork (https://github.com/ROCm/vllm) will require a clear previously communicated exception

Signed-off-by: fsx950223 <fsx950223@outlook.com>

…t#18620) Signed-off-by: Zerohertz <ohg3417@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…m-project#18200) Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: mgoin <mgoin64@gmail.com>

…-project#17945)" (vllm-project#18593) Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: David Xia <david@davidxia.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…ar (vllm-project#18627) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Rui Qiao <ruisearch42@gmail.com> Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn> Co-authored-by: Rui Qiao <ruisearch42@gmail.com>

…lm-project#18537) Signed-off-by: Huy Do <huydhn@gmail.com>

Signed-off-by: mgoin <mgoin64@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…ntrol blockscale tensor allocation (vllm-project#18160) Signed-off-by: Pavani Majety <pmajety@nvidia.com>

vllm-project#18454) Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com> Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>

Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

…llm-project#18623) Signed-off-by: Mathieu Bordere <mathieu@letmetweakit.com>

Signed-off-by: wenhuach21 <wenhua.cheng@intel.com>

…mance (vllm-project#18424) Signed-off-by: qizixi <qizixi@meta.com>

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

…lGeneration) (vllm-project#18647)

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

…oding (vllm-project#18668) Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

…ules.deepseek-ai.DeepSeek-V2-Lite... (vllm-project#18640) Signed-off-by: Seiji Eicher <seiji@anyscale.com>

Signed-off-by: Andy Xie <andy.xning@gmail.com>

…roject#14668) Signed-off-by: noemotiovon <757486878@qq.com>

…oject#18655) Signed-off-by: Mengqing Cao <cmq0113@163.com> Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <2037008807@qq.com>

…t#18499) Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Signed-off-by: mgoin <mgoin64@gmail.com>

Signed-off-by: Andy Xie <andy.xning@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

…nt (vllm-project#18674) Signed-off-by: zzzyq <zhangyuqi94@gmail.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

…hashing (vllm-project#18682) Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>

…18683) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <2037008807@qq.com>

…example (vllm-project#18644) Signed-off-by: zhaohaidao <zhaohaidao2008@hotmail.com> Signed-off-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com> Co-authored-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com>

…handler assignment (vllm-project#18690) Signed-off-by: googs1025 <googs1025@gmail.com>

Signed-off-by: Max de Bayser <mbayser@br.ibm.com> Signed-off-by: Max de Bayser <maxdebayser@gmail.com>

Signed-off-by: fsx950223 <fsx950223@outlook.com>

github-actions · 2025-05-29T06:29:49Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

fsx950223 and others added 30 commits May 8, 2025 05:34

integrate aiter

d483fc2

Signed-off-by: fsx950223 <fsx950223@outlook.com>

add env variable

4f85566

Signed-off-by: fsx950223 <fsx950223@outlook.com>

rename function

ae85e79

Signed-off-by: fsx950223 <fsx950223@outlook.com>

optimize kernels with small query lens

87ea0ba

Signed-off-by: fsx950223 <fsx950223@outlook.com>

change condition

db4bc55

Signed-off-by: fsx950223 <fsx950223@outlook.com>

Merge remote-tracking branch 'upstream/main' into fa_upstream

efe59bd

Signed-off-by: fsx950223 <fsx950223@outlook.com>

add rocm aiter backend

40654e4

Signed-off-by: fsx950223 <fsx950223@outlook.com>

new fa impl

8dd236d

Signed-off-by: fsx950223 <fsx950223@outlook.com>

[Doc] Fix markdown list indentation for MkDocs rendering (vllm-projec…

5221815

…t#18620) Signed-off-by: Zerohertz <ohg3417@gmail.com>

[Doc] Use a different color for the announcement (vllm-project#18616)

022d8ab

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Refactor pplx init logic to make it modular (prepare for deepep) (vll…

6a7988c

…m-project#18200) Signed-off-by: youkaichao <youkaichao@gmail.com>

Fix figures in design doc (vllm-project#18612)

3d28ad3

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

[Docs] Change mkdocs to not use directory urls (vllm-project#18622)

9520a98

Signed-off-by: mgoin <mgoin64@gmail.com>

[v1] Redo "Support multiple KV cache groups in GPU model runner (vllm…

6550114

…-project#17945)" (vllm-project#18593) Signed-off-by: Chen Zhang <zhangch99@outlook.com>

[Doc] fix list formatting (vllm-project#18624)

8ddd1cf

Signed-off-by: David Xia <david@davidxia.com>

[Doc] Fix top-level API links/docs (vllm-project#18621)

273cb3b

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Doc] Avoid documenting dynamic / internal modules (vllm-project#18626)

15b45ff

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Doc] Fix broken links and unlinked docs, add shortcuts to home sideb…

371f7e4

…ar (vllm-project#18627) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[V1] Support Deepseek MTP (vllm-project#18435)

2628a69

Signed-off-by: Rui Qiao <ruisearch42@gmail.com> Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn> Co-authored-by: Rui Qiao <ruisearch42@gmail.com>

Use prebuilt FlashInfer x86_64 PyTorch 2.7 CUDA 12.8 wheel for CI (vl…

1645b60

…lm-project#18537) Signed-off-by: Huy Do <huydhn@gmail.com>

[CI] Enable test_initialization to run on V1 (vllm-project#16736)

0ddf88e

Signed-off-by: mgoin <mgoin64@gmail.com>

[Doc] Update references to doc files (vllm-project#18637)

7d92164

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to co…

f203673

…ntrol blockscale tensor allocation (vllm-project#18160) Signed-off-by: Pavani Majety <pmajety@nvidia.com>

[Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (

4fc1bf8

vllm-project#18454) Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com> Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>

[Bugfix][Nixl] Fix Preemption Bug (vllm-project#18631)

2b10ba7

Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

config.py: Clarify that only local GGUF checkpoints are supported. (v…

45ab403

…llm-project#18623) Signed-off-by: Mathieu Bordere <mathieu@letmetweakit.com>

FIX MOE issue in AutoRound format (vllm-project#18586)

ec82c3e

Signed-off-by: wenhuach21 <wenhua.cheng@intel.com>

[V1][Spec Decode] Small refactors to improve eagle bookkeeping perfor…

d55e446

…mance (vllm-project#18424) Signed-off-by: qizixi <qizixi@meta.com>

[Frontend] improve vllm serve --help display (vllm-project#18643)

441dc63

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[Model] Add support for Qwen2.5-Omni-7B-AWQ (Qwen2_5OmniForConditiona…

a859320

…lGeneration) (vllm-project#18647)

aarnphm and others added 28 commits May 24, 2025 08:04

[CI] enforce import regex instead of re (vllm-project#18665)

2807271

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

fix(regression): clone from reference items (vllm-project#18662)

9ea7f1a

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

[CI/Build] fix permission denied issue (vllm-project#18645)

b554ab7

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[BugFix][Spec Decode] Improve Prefix Caching Logic in Speculative Dec…

6825d9a

…oding (vllm-project#18668) Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

[V1] Fix _pickle.PicklingError: Can't pickle <class 'transformers_mod…

7891fdf

…ules.deepseek-ai.DeepSeek-V2-Lite... (vllm-project#18640) Signed-off-by: Seiji Eicher <seiji@anyscale.com>

[MISC] correct signature for LoaderFunction (vllm-project#18670)

6c6dcd8

Signed-off-by: Andy Xie <andy.xning@gmail.com>

[Misc]Replace cuda hard code with current_platform in Ray (vllm-p…

cebc22f

…roject#14668) Signed-off-by: noemotiovon <757486878@qq.com>

[Misc][ModelScope] Change to use runtime VLLM_USE_MODELSCOPE (vllm-pr…

6ab681b

…oject#18655) Signed-off-by: Mengqing Cao <cmq0113@163.com> Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <2037008807@qq.com>

[VLM] Initialize video input support for InternVL models (vllm-projec…

75f8175

…t#18499) Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Speed up the kernels/quantization/ tests (vllm-project#18669)

6393454

Signed-off-by: mgoin <mgoin64@gmail.com>

[BUGFIX] catch subclass first for try...except (vllm-project#18672)

44073a7

Signed-off-by: Andy Xie <andy.xning@gmail.com>

[Misc] Reduce logs on startup (vllm-project#18649)

503f848

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[doc] fix broken links (vllm-project#18671)

624b77a

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[doc] improve readability (vllm-project#18675)

279f854

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[Bugfix] Fix cpu usage and cache hit stats reporting on cpu environme…

f2faac7

…nt (vllm-project#18674) Signed-off-by: zzzyq <zhangyuqi94@gmail.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

[CI/build] fix no regex (vllm-project#18676)

35be8fa

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[Misc] small improve (vllm-project#18680)

3a886bd

Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>

[Bugfix] Fix profiling dummy data for Pixtral (vllm-project#18677)

57fd13a

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[Core][Multimodal] Convert PIL Image to array without data copy when …

6071e98

…hashing (vllm-project#18682) Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>

[CI/Build][Doc] Update gte-Qwen2-1.5B-instruct usage (vllm-project#…

fba0642

…18683) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <2037008807@qq.com>

[Misc] Fixed the abnormally high TTFT issue in the PD disaggregation …

8820821

…example (vllm-project#18644) Signed-off-by: zhaohaidao <zhaohaidao2008@hotmail.com> Signed-off-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com> Co-authored-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com>

refactor: simplify request handler, use positive condition check for …

abd4030

…handler assignment (vllm-project#18690) Signed-off-by: googs1025 <googs1025@gmail.com>

[Bugfix] Fix the lm_head in gpt_bigcode in lora mode (vllm-project#6357)

561b77a

Signed-off-by: Max de Bayser <mbayser@br.ibm.com> Signed-off-by: Max de Bayser <maxdebayser@gmail.com>

update api

e995fdc

Signed-off-by: fsx950223 <fsx950223@outlook.com>

optimize performance

a501ff0

Signed-off-by: fsx950223 <fsx950223@outlook.com>

merge

3ad9e3a

Signed-off-by: fsx950223 <fsx950223@outlook.com>

remove try catch

05e460f

Signed-off-by: fsx950223 <fsx950223@outlook.com>

clean code

0a337a6

Signed-off-by: fsx950223 <fsx950223@outlook.com>

fsx950223 closed this May 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fa upstream3#560

Fa upstream3#560
fsx950223 wants to merge 68 commits into
0610_release_testingfrom
fa_upstream3

fsx950223 commented May 29, 2025 •

edited by github-actions Bot

Loading

Uh oh!

github-actions Bot commented May 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Uh oh!

Conversation

fsx950223 commented May 29, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

fsx950223 commented May 29, 2025 •

edited by github-actions Bot

Loading