Skip to content

Commit 1a0aca1

Browse files
committed
feat(cambricon): blacklist flag_gems empty via built-in config
Add vllm_fl/dispatch/config/cambricon.yaml so the `empty` operator blacklist is applied automatically on Cambricon MLU, instead of requiring the VLLM_FL_FLAGOS_BLACKLIST env var at launch. flag_gems `empty` derives its Triton launch grid from the element count (e.g. 607744 for the Qwen3 vocab embedding), which exceeds the MLU Triton grid limit of 65535 and raises OutOfResources. The config falls the op back to torch_mlu. The file is named `cambricon.yaml` to match current_platform.vendor_name ("cambricon"), which is how get_config_path() resolves the platform config (Priority 3). Verified E2E on MLU590: serve reaches startup complete and inference returns coherent output with no env var set. This PR was written in part with the assistance of generative AI.
1 parent 7f6f35e commit 1a0aca1

1 file changed

Lines changed: 20 additions & 0 deletions

File tree

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# vLLM-FL Dispatch Configuration for Cambricon MLU (torch_mlu)
2+
# Auto-loaded when running on Cambricon MLU hardware (vendor_name == "cambricon").
3+
#
4+
# There is no dedicated vendor/cambricon dispatch backend; ops route through
5+
# FlagOS (flag_gems, Triton). Only per-op exceptions are listed below.
6+
7+
# Preferred default backend type: flagos, vendor, reference
8+
prefer: flagos
9+
10+
# Strict Mode:
11+
# true = Raise an error immediately on failure; do not attempt other backends.
12+
# false = Attempt the next available backend in sequence upon failure (Default).
13+
strict: false
14+
15+
# FlagOS operator blacklist
16+
flagos_blacklist:
17+
# flag_gems `empty` computes a Triton launch grid from the element count
18+
# (e.g. 607744 for the Qwen3 vocab embedding), which exceeds the MLU Triton
19+
# grid limit of 65535 and raises OutOfResources. Fall back to torch_mlu.
20+
- empty

0 commit comments

Comments
 (0)