[hot-fix] fix GroupedTopKRouterFL routing and MLA decode method name - #352
Open
tonyh168 wants to merge 2 commits into
Open
[hot-fix] fix GroupedTopKRouterFL routing and MLA decode method name#352tonyh168 wants to merge 2 commits into
tonyh168 wants to merge 2 commits into
Conversation
added 2 commits
August 7, 2026 15:50
GroupedTopKRouterFL._compute_routing called self._valid_grouping(), which does not exist on upstream GroupedTopKRouter (it uses an inner valid_grouping() closure), raising AttributeError during grouped top-k routing. Port the inline closure from v0.3.0-dev.
vllm 0.20.0 renamed the MLACommonImpl decode override from _forward_decode to forward_mqa (same signature). MLAFLImpl still defined _forward_decode, so its implementation was never dispatched and the abstract forward_mqa raised NotImplementedError at decode. Rename to match the base class.
|
loopyt seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
基于
v0.2.0tag 的 hotfix,修复两个运行时 bug。共两个 commit,各修一处。Bug 1 — GroupedTopKRouterFL 路由报错 (
4b6c1b2)vllm_fl/ops/fused_moe/router.py中GroupedTopKRouterFL._compute_routing调用了
self._valid_grouping(router_logits),但上游GroupedTopKRouter并没有
_valid_grouping方法 —— 它在_compute_routing内部用一个局部闭包valid_grouping()做校验。因此启用 grouped top-k 路由(DeepSeek-V2/V3、GLM-MoE、混元等模型)时会抛
AttributeError。修复:移植 v0.3.0-dev 的做法,用内联的
valid_grouping()闭包替换该调用(判断逻辑:
num_experts > num_expert_group且能被整除)。仅移植该最小修复,不含 v0.3.0-dev 中无关的
call_op→CachedOpdispatch 重构。```python
def valid_grouping() -> bool:
num_experts = router_logits.shape[-1]
if num_experts <= self.num_expert_group:
return False
return num_experts % self.num_expert_group == 0
if not valid_grouping():
...
```
Bug 2 — MLA decode 方法签名对不上 (
2bac207)vllm_fl/dispatch/backends/flaggems/impl/mla.py中MLAFLImpl继承自MLACommonImpl。vllm 0.20.0 把基类的 decode 覆写方法从_forward_decode重命名为
forward_mqa(参数签名完全相同)。FL 侧仍定义_forward_decode,导致该实现永远不会被调用,decode 时命中基类抽象的
forward_mqa→ 抛NotImplementedError。修复:将
MLAFLImpl._forward_decode重命名为forward_mqa,方法体不变。(metax 后端用的是自带 vendored 的 MLA,不受影响,未改动。)
Test
python -m py_compilerouter.py中不再存在self._valid_grouping调用mla.py中 decode 方法名为forward_mqa