[ascend]: add qwen3next

Author: Yanguan

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,8 @@
+### PR Category
+<!-- One of [Core | Vendor | OP | Tools | Others ] -->
+
+### PR Types
+<!-- One of [ User Experience | New Features | Bug Fixes | Improvements | Performance | Breaking Change| Deprecations | Test Case | Docs | Others ] -->
+
+### PR Description
+<!-- Describe what you’ve done -->

.github/labeler.yml

@@ -0,0 +1,35 @@
+# PR Labeler configuratoin file
+# Automatically add labels based on modified file paths
+
+docs:
+  - changed-files:
+      - any-glob-to-any-file: '**/*.md'
+
+ci:
+  - changed-files:
+      - any-glob-to-any-file:
+          - '.github/**/*'
+          - '.pre-commit-config.yaml'
+
+tests:
+  - changed-files:
+      - any-glob-to-any-file: 'tests/**/*'
+
+core:
+  - changed-files:
+      - any-glob-to-any-file: 'vllm_fl/**/*'
+
+examples:
+  - changed-files:
+      - any-glob-to-any-file: 'examples/**/*'
+
+benchmarks:
+  - changed-files:
+      - any-glob-to-any-file: 'benchmarks/**/*'
+
+build:
+  - changed-files:
+      - any-glob-to-any-file:
+          - 'setup.py'
+          - 'requirements*.txt'
+          - 'pyproject.toml'

.github/workflows/labeler.yml

@@ -0,0 +1,24 @@
+name: "Pull Request Labeler"
+
+on:
+  pull_request_target:
+    types: [opened, synchronize, reopened]
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  label:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Apply labels
+        uses: actions/labeler@v5
+        continue-on-error: true  # Don't fail if config not yet on main branch
+        with:
+          repo-token: "${{ secrets.GITHUB_TOKEN }}"
+          sync-labels: true

.gitignore

@@ -0,0 +1,8 @@
+*.egg-info
+__pycache__/
+build/
+
+# Coverage
+.coverage
+.coverage.*
+htmlcov/

.pre-commit-config.yaml

@@ -0,0 +1,21 @@
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v5.0.0
+    hooks:
+    -   id: check-merge-conflict
+    -   id: check-symlinks
+    -   id: detect-private-key
+    -   id: end-of-file-fixer
+    -   id: trailing-whitespace
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  rev: v0.14.0
+  hooks:
+  - id: ruff-check
+    args: ["--ignore=E731", --output-format, github, --fix]
+    # E731 : Do not assign a lambda expression, use a def
+  - id: ruff-format
+# - repo: https://github.com/crate-ci/typos
+#   rev: v1.38.1
+  # hooks:
+  # - id: typos
+  #   args: [--force-exclude]

README.md

@@ -5,65 +5,73 @@ A vLLM plugin built on the FlagOS unified multi-chip backend.
 
 ### Setup
 
-0. Install vllm from the official [v0.11.0](https://github.com/vllm-project/vllm/tree/v0.11.0) (optional if the correct version is installed) or from the fork [vllm-FL](https://github.com/flagos-ai/vllm-FL).
+0. Install vllm from the official [v0.13.0](https://github.com/vllm-project/vllm/tree/v0.13.0) (optional if the correct version is installed) or from the fork [vllm-FL](https://github.com/flagos-ai/vllm-FL).
 
 
-1. Install FlagGems
+1. Install vllm-plugin-FL
 
-    1.1 Install Build Dependencies
+    1.1 Clone the repository:
+
+    ```sh
+    git clone https://github.com/flagos-ai/vllm-plugin-FL
+    ```
+
+    1.2 install
+    ```sh
+    cd vllm-plugin-FL
+    pip install -r requirements.txt
+    pip install --no-build-isolation .
+    # or editble install
+    pip install --no-build-isolation -e .
+    ```
+
+2. Install [FlagGems](https://github.com/flagos-ai/FlagGems/blob/master/docs/getting-started.md#quick-installation)
+
+    2.1 Install Build Dependencies
 
     ```sh
     pip install -U scikit-build-core==0.11 pybind11 ninja cmake
     ```
 
-    1.2 Installation FlagGems
+    2.2 Installation FlagGems
 
-    ```shell
+    ```sh
     git clone https://github.com/flagos-ai/FlagGems
     cd FlagGems
     pip install --no-build-isolation .
     # or editble install
     pip install --no-build-isolation -e .
     ```
 
-2. Install FlagCX
+3. Install [FlagCX](https://github.com/flagos-ai/FlagCX/blob/main/docs/getting_started.md#build-and-installation)
 
-    2.1 Clone the repository:
+    3.1 Clone the repository:
     ```sh
     git clone https://github.com/flagos-ai/FlagCX.git
     cd FlagCX
-    git checkout v0.3.0
+    git checkout -b v0.9.0
+    git submodule update --init --recursive
     ```
 
-    2.2 Build the library with different flags targeting to different platforms:
+    3.2 Build the library with different flags targeting to different platforms:
     ```sh
     make USE_NVIDIA=1
     ```
 
-    2.3 Set environment
+    3.3 Set environment
     ```sh
-    export FLAGCX_PATH="$pwd"
+    export FLAGCX_PATH="$PWD"
     ```
 
-    2.4 Installation FlagGems
+    3.4 Installation FlagCX
     ```sh
     cd plugin/torch/
-    python setup.py develop --adaptor nvidia/ascend
-    ```
-
-3. Install vllm-plugin-fl
-
-    3.1 Clone the repository:
-
-    ```sh
-    git clone https://github.com/flagos-ai/vllm-plugin-FL
+    FLAGCX_ADAPTOR=[xxx] pip install . --no-build-isolation
+    # or editable install
+    FLAGCX_ADAPTOR=[xxx] pip install -e . --no-build-isolation
     ```
+    Note: [xxx] should be selected according to the current platform, e.g., nvidia, ascend, etc.
 
-    3.2 install
-    ```sh
-    cd vllm-plugin-fl
-    pip install --no-build-isolation -e .
-    ```
 
 If there are multiple plugins in the current environment, you can specify use vllm-plugin-fl via VLLM_PLUGINS='fl'.
 
@@ -92,3 +100,19 @@ if __name__ == '__main__':
         generated_text = output.outputs[0].text
         print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
 ```
+
+## Advanced use
+
+For dispatch environment variable usage, see [environment variables usage](./vllm_fl/dispatch/README.md#environment-variables).
+
+### Using Cuda Communication library
+If you want to use the original Cuda Communication, you can unset the following environment variables.
+```sh
+unset FLAGCX_PATH
+```
+
+### Using native CUDA operators
+If you want to use the original CUDA operators, you can set the following environment variables.
+```sh
+export USE_FLAGGEMS=0
+```

benchmarks/README.md

@@ -1,30 +1,83 @@
-To use the benchmark_throughput_flagos feature from vllm-plugin-fl, you must first complete the following preliminary steps:
+## Benchmark Overview
 
-1. Start an LLM inference service compliant with the OpenAI API protocol using the --served-model-name Qwen3-Next argument, or use a different name and update the string on line 11 of benchmark_throughput_flagos.py to match your chosen --served-model-name exactly (character-for-character).
+This directory provides two workflows:
+- `benchmark_throughput_flagos.py`: throughput benchmarking for a served model.
+- `benchmark_throughput_autotune.py`: auto-tune FlagGems operator selection by throughput.
 
-2. Run the benchmark_throughput_flagos script on the same machine where the inference service is hosted.
+---
 
-3. Ensure the host machine has stable global network bandwidth of at least 10 Mbps, with 100 Mbps recommended for reliable benchmarking.
+## benchmark_throughput_flagos.py
 
-4. The benchmark_throughput_flagos feature has been validated in environments using vLLM versions 0.11.0 and 0.12.0. Using higher or lower vLLM versions—or serving the model with alternative frameworks such as SGLang—may result in compatibility issues.
-
-Once the above prerequisites are met, you can simply run the following command from the directory containing this file: 
+### Note
+- Start an OpenAI-compatible inference service with `--served-model-name Qwen3-Next`.
+  If you use a different name, update the string in `benchmark_throughput_flagos.py`.
+- Run the benchmark on the same host as the service.
 
+### Run
 ```bash
 python3 benchmark_throughput_flagos.py
 ```
 
-If the command runs successfully, it will generate a directory named vllm_bench_logs in the current working directory. After the execution completes, follow these two steps to verify the benchmark ran correctly and to obtain the performance evaluation results: 
-
-1. Check for failed requests by running the following command in the current directory:
+### Verify Results
+1) Check failed requests:
 ```bash
 grep "Fail" -rn vllm_bench_logs
 ```
+All matches should show `Fail: 0`.
 
-All matching lines should report zero failed requests (i.e., Fail: 0).
-
-2. Generate performance statistics by executing:
+2) Generate statistics:
 ```bash
 python3 benchmark_throughput_flagos_statistics.py
 ```
-This will output the final performance evaluation results based on the collected logs.
+
+---
+
+## benchmark_throughput_autotune.py
+
+
+### Command
+```bash
+python benchmarks/benchmark_throughput_autotune.py [vllm args] [autotune options]
+```
+
+### vLLM Args
+All arguments are passed directly to `vllm bench throughput`.
+
+### Autotune Options
+- `--background` (true/false): run in background mode, default `false`.
+- `--ops`: comma-separated list of operator names to tune. If empty, auto-discovers ops.
+- `--num-runs`: number of runs per configuration, default `2` (uses the second round to skip warmup).
+- `--csv-path`: output CSV filename, default `history.csv` under the run directory.
+
+Example:
+```bash
+python benchmarks/benchmark_throughput_autotune.py \
+  --model /models/Qwen3-Next-80B-A3B-Instruct \
+  --tensor-parallel-size 4 \
+  --dataset-name random \
+  --input-len 6144 \
+  --output-len 1024 \
+  --num-prompts 1000 \
+  --max-num-batched-tokens 16384 \
+  --max-num-seqs 2048 \
+  --load-format "dummy" \
+  --gpu-memory-utilization 0.85 \
+  --compilation-config '{"cudagraph_mode": "FULL_AND_PIECEWISE"}' \
+  --background true
+```
+
+### Environment Variables
+See [environment variables usage](../vllm_fl/dispatch/README.md#environment-variables) for the full dispatch and FlagGems configuration reference.
+
+### Outputs
+Each run creates a directory under `autotune_logs/autotune_xxx` with:
+- `autotune.log`: full run log when setting `background` as `true`
+- `history.csv`: throughput results for all rounds
+- `autotune_ops.yaml`: final op list used for tuning
+- `autotune_configs/`: per-round config snapshots
+- `best_config.yaml`: selected best config
+
+### Notes
+- Round 1 runs baseline throughput without FlagGems.
+- Round 2 benchmarks each op in isolation (whitelist).
+- Round 3 validates the best-performing set.