Skip to content

Commit 2bac207

Browse files
author
loopyt
committed
fix(mla): rename _forward_decode to forward_mqa for vllm 0.20.0
vllm 0.20.0 renamed the MLACommonImpl decode override from _forward_decode to forward_mqa (same signature). MLAFLImpl still defined _forward_decode, so its implementation was never dispatched and the abstract forward_mqa raised NotImplementedError at decode. Rename to match the base class.
1 parent 4b6c1b2 commit 2bac207

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • vllm_fl/dispatch/backends/flaggems/impl

vllm_fl/dispatch/backends/flaggems/impl/mla.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,7 @@ def _flash_attn_varlen_diff_headdims(
119119
return attn_out, lse
120120
return attn_out
121121

122-
def _forward_decode(
122+
def forward_mqa(
123123
self,
124124
q: Union[torch.Tensor, tuple[torch.Tensor, torch.Tensor]],
125125
kv_c_and_k_pe_cache: torch.Tensor,

0 commit comments

Comments
 (0)