Support kunlunxin backend by luoyc123 · Pull Request #268 · flagos-ai/vllm-plugin-FL

luoyc123 · 2026-07-06T07:51:35Z

PR Category

Others

PR Type

New Features

Description

Based on vLLM 0.20.2 (empty), adapted for vllm-plugin-FL with support for the Kunlunxin backend.The Qwen3.6-35B-A3B and Qwen3.6-27B models have been verified, the test includes high-concurrency input of text and images.
Test command：
export FLAGCX_PATH=/workspace/FlagCX
export USE_RESHAPE_AND_CACHE_FLASH=1
vllm serve /workspace/models/Qwen3.6-27B
--served-model-name qwen3
--tensor-parallel-size 4
--max-model-len 16384
--reasoning-parser qwen3
--block-size 128
--gpu-memory-utilization 0.8
--enforce-eager
--port 8100

Verified results：

Changes

Add Kunlunxin backend support.

Testing

Checklist

I have run the existing tests and they pass
I have added tests for my changes (if applicable)
I have updated the documentation (if applicable)

+        output_final_state,
+        cu_seqlens,
+        use_qk_l2norm_in_kernel,
+    )


+            stacklevel=2,
+        )
+        q, k, v, beta, g = map(


+    global _patched
+    if _patched:
+        return
+    _patched = True


+        key_cache: torch.Tensor,
+        value_cache: torch.Tensor,
+        slot_mapping: torch.Tensor,
+        k_max: torch.Tensor | None = None,
+        v_max: torch.Tensor | None = None,
+        quant_mode: int = 0,
+        force_sdnn: bool = False,
+        BLHD_LAYOUT: bool = True) -> int:
+        """
+        reshape and store key and value in cache_key, cache_value, respectively.
+


+        max_query_len = common_attn_metadata.max_query_len
+        block_table_tensor = common_attn_metadata.block_table_tensor
+        slot_mapping = common_attn_metadata.slot_mapping
+        seq_lens_cpu = common_attn_metadata.seq_lens_cpu


+try:
+    import torch_xmlir  # noqa: F401
+    import xtorch_ops
+    _KUNLUNXIN_OPS_AVAILABLE = True


+import logging
+import math
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any, ClassVar, Dict, List, Optional, Tuple, Type


+    global _patches_applied
+    if _patches_applied:
+        return
+    _patches_applied = True


+    is_conv_state_dim_first = gdn_mod.is_conv_state_dim_first
+    causal_conv1d_fn = gdn_mod.causal_conv1d_fn
+    causal_conv1d_update = gdn_mod.causal_conv1d_update
+    fused_post_conv_prep = gdn_mod.fused_post_conv_prep


cyber-pioneer · 2026-07-28T11:02:06Z

what's xtorch_ops ?

cyber-pioneer · 2026-07-29T04:54:58Z

add pr description

- fused_moe_utils.py: rollback to correct-container version, remove broken is_cuda() guard so XPU takes fast path with proper return statement - worker.py: rollback to restore NPU compatibility code (empty_cache patch, profile_run skip, _dummy_run skip)

…rmatting - Remove unused imports: Enum, rocm_aiter_ops, MoEBackend, has_flashinfer_cutlass_fused_moe, triton (only tl is used) - Restore multi-line import style for oracle.unquantized, fused_moe, fused_moe.utils - Move current_platform import back to original position - Restore multi-line function signature for select_unquantized_moe_backend_oot - Remove trailing whitespace on use_flaggems() condition line - Restore blank lines between top-level definitions

+    # Step 2: force-import utils.py NOW (it loads safely with the wrapper)
+    try:
+        import vllm.model_executor.layers.fla.ops.utils  # noqa: F401
+    except Exception:


+
+    # Apply Sunrise/PTPU monkey-patches if running on PTPU.
+    if current_platform.device_type == "ptpu":
+        from vllm_fl.dispatch.backends.vendor.sunrise.patch import apply_sunrise_patches


CherryLemon · 2026-08-19T06:57:55Z

+
+    # 1.3: Validate activation function
+    SUPPORTED_ACTIVATIONS = ["silu", "gelu", "relu", "gelu_no_mul", "silu_no_mul"]
+    if activation not in SUPPORTED_ACTIVATIONS:


In vLLM 0.20.2, MoEActivation is a plain Enum, FusedMoE stores the parsed enum, and the unquantized path passes layer.activation directly to the kernel. Therefore this string comparison rejects even MoEActivation.SILU. The list also uses relu, while vLLM defines RELU2 = "relu2". Please normalize enum values before validation and align the supported names with MoEActivation.

CherryLemon · 2026-08-19T06:57:55Z

+        )
+
+    # Reject varlen mode with query_start_loc: not supported by native kernel
+    if query_start_loc is not None and max_query_len > 0:


This guard rejects only when both arguments are set. If query_start_loc is provided alone, or max_query_len > 0 alone, the unsupported value is still silently ignored. Please reject either non-default argument (use or) or validate the supported combinations explicitly.

luoyc123 added 2 commits January 28, 2026 16:29

add MimiCMP files

8742e43

delete MiniCMP

5e6c189

github-actions Bot added the core label Jul 6, 2026

github-advanced-security AI found potential problems Jul 6, 2026

View reviewed changes

luoyc123 force-pushed the support_kunlunxin_backend_2 branch from b3f9ffc to 51573ff Compare July 6, 2026 07:56

ceci3 reviewed Jul 7, 2026

View reviewed changes

Comment thread vllm_fl/ops/fused_moe/fused_moe_utils.py Outdated

Comment thread vllm_fl/ops/fused_moe/fused_moe_utils.py Outdated

Comment thread vllm_fl/__init__.py Outdated

github-advanced-security AI found potential problems Jul 22, 2026

View reviewed changes

Comment thread vllm_fl/__init__.py Fixed

luoyc123 changed the title ~~Support kunlunxin backend~~ [WIP] Support kunlunxin backend Jul 31, 2026

luoyc123 force-pushed the support_kunlunxin_backend_2 branch from 8078584 to b38390f Compare August 10, 2026 01:36

luoyc123 and others added 7 commits August 10, 2026 09:55

Merge branch 'flagos-ai:main' into main

be1a56e

feat(kunlunxin): add Kunlunxin XPU backend

56c0d99

Addressing the issue of Chinese language deterioration

ffde7c3

Switch CPU computation to XPU computation.

a04fb18

bugfix of triton kernel support on kunlunxin

f8952b4

Reply to delete code

c5b373f

Remove unnecessary modified code

511a39a

luoyc123 force-pushed the support_kunlunxin_backend_2 branch from b38390f to 511a39a Compare August 10, 2026 02:12

support flagos blacklist

b4460b7

cyber-pioneer reviewed Aug 10, 2026

View reviewed changes

Comment thread vllm_fl/worker/worker.py Outdated

cyber-pioneer reviewed Aug 10, 2026

View reviewed changes

Comment thread vllm_fl/__init__.py Outdated

cyber-pioneer reviewed Aug 10, 2026

View reviewed changes

Comment thread vllm_fl/ops/fused_moe/fused_moe_utils.py

luoyc123 force-pushed the support_kunlunxin_backend_2 branch 2 times, most recently from bb87fef to 0b5968a Compare August 11, 2026 07:50

luoyc123 force-pushed the support_kunlunxin_backend_2 branch from 0b5968a to 8318c25 Compare August 11, 2026 07:54

cyber-pioneer reviewed Aug 11, 2026

View reviewed changes

Comment thread vllm_fl/__init__.py

Comment thread vllm_fl/ops/fused_moe/fused_moe_utils.py Outdated

luoyc123 added 3 commits August 12, 2026 12:35

Align with main

a38284c

Remove unnecessary code

f0b2fc5

Restrict the platforms where the reduction operator can be used

bf5c43b

luoyc123 changed the title ~~[WIP] Support kunlunxin backend~~ Support kunlunxin backend Aug 12, 2026

Merge branch 'main' into support_kunlunxin_backend_2

c6f2eb6

github-advanced-security AI found potential problems Aug 12, 2026

View reviewed changes

Comment thread vllm_fl/dispatch/backends/vendor/kunlunxin/patches/patch_fla_utils.py

# Step 2: force-import utils.py NOW (it loads safely with the wrapper)

try:

import vllm.model_executor.layers.fla.ops.utils # noqa: F401

except Exception:

ceci3 reviewed Aug 12, 2026

View reviewed changes

Comment thread vllm_fl/ops/fused_moe/fused_moe_utils.py

Comment thread vllm_fl/__init__.py Outdated

Comment thread vllm_fl/platform.py

Comment thread vllm_fl/dispatch/backends/vendor/kunlunxin/patches/patch_fla_utils.py

github-advanced-security AI found potential problems Aug 17, 2026

View reviewed changes

Comment thread vllm_fl/dispatch/config/utils.py Fixed

Fix log flooding

e627bf8

luoyc123 force-pushed the support_kunlunxin_backend_2 branch from 08fb5ac to e627bf8 Compare August 17, 2026 06:48

luoyc123 and others added 3 commits August 17, 2026 15:52

Merge branch 'main' into support_kunlunxin_backend_2

79396bd

Refactor: extract _init_vendor_device for vendor-specific early patches

6ffa703

Merge branch 'main' into support_kunlunxin_backend_2

33abe8f

cyber-pioneer previously approved these changes Aug 18, 2026

View reviewed changes

CherryLemon reviewed Aug 18, 2026

View reviewed changes

ceci3 reviewed Aug 18, 2026

View reviewed changes

fix: address code review comments for kunlunxin vendor backend

c821a04

luoyc123 dismissed cyber-pioneer’s stale review via c821a04 August 19, 2026 01:52

github-advanced-security AI found potential problems Aug 19, 2026

View reviewed changes

Comment thread vllm_fl/ops/custom_ops.py

# Apply Sunrise/PTPU monkey-patches if running on PTPU.

if current_platform.device_type == "ptpu":

from vllm_fl.dispatch.backends.vendor.sunrise.patch import apply_sunrise_patches

Merge branch 'main' into support_kunlunxin_backend_2

d52385a

CherryLemon reviewed Aug 19, 2026

View reviewed changes

ceci3 approved these changes Aug 19, 2026

View reviewed changes

cyber-pioneer approved these changes Aug 19, 2026

View reviewed changes

cyber-pioneer merged commit 38e7dbc into flagos-ai:main Aug 19, 2026
96 of 97 checks passed

Uh oh!

Conversation

luoyc123 commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Category

PR Type

Description

Changes

Testing

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cyber-pioneer commented Jul 28, 2026

Uh oh!

cyber-pioneer commented Jul 29, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CherryLemon Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CherryLemon Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

luoyc123 commented Jul 6, 2026 •

edited

Loading

CherryLemon Aug 19, 2026 •

edited

Loading