[cherry-pick] cherry pick commit_id bbb766d cefbff6 af71c54 3729890 28219b5 cbf88e2 from main to 0.1.0 (#116)

### PR Category
<!-- One of [Core | Vendor | OP | Tools | Others] -->
Others

### PR Type
<!-- One of [User Experience | New Features | Bug Fixes | Improvements |
Performance | Breaking Change | Deprecations | Test Case | Docs |
Others] -->
Others

### Description
<!-- Describe what this PR does and why. -->
cherry pick commit_id bbb766d cefbff6 af71c54 3729890 28219b5 cbf88e2
from main to 0.1.0

### Related Issues
<!-- Link any related issues: Fixes #issue, Closes #issue, or Related to
#issue -->

### Changes
<!-- List the key changes made in this PR. -->


### Testing
<!-- How has this change been tested? Include test commands, hardware
used, etc. -->
-

### Checklist
- [ ] I have run the existing tests and they pass
- [ ] I have added tests for my changes (if applicable)
- [ ] I have updated the documentation (if applicable)

---------

Co-authored-by: xiangbin <74005582+li199959@users.noreply.github.com>
Co-authored-by: keennddyl <keennddy@foxmail.com>
Co-authored-by: jiamingwang-mt <jiaming.wang@mthreads.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: keennddyl <keenndd@foxmail.com>
Co-authored-by: hozier <hozier@mthreads.com>
Co-authored-by: cyber-pioneer <116002591+cyber-pioneer@users.noreply.github.com>
Co-authored-by: XMing <xmhubj@gmail.com>
Co-authored-by: xin2an <yuxinan@perfxlab.com>

Author: 10 people

.github/ISSUE_TEMPLATE/blank.md

@@ -5,4 +5,3 @@ title: ''
 labels: ''
 assignees: ''
 ---
-

.github/configs/cuda.yml

@@ -20,6 +20,7 @@ container_volumes:
 
 # Container options (hardware-specific settings)
 container_options: >-
+  --privileged
   --gpus all
   --shm-size=500g
   --hostname vllm-plugin-fl

.github/scripts/cuda/setup.sh

@@ -83,8 +83,10 @@ jobs:
 
       - name: Upload coverage report to FlagCICD
         if: inputs.upload_coverage && always()
-        uses: flagos-ai/FlagOps/actions/post-pytest-report@v1
+        uses: flagos-ai/FlagOps/actions/post-pytest-report@v2
+        continue-on-error: true
         with:
-          backend_url: 'http://10.1.4.167:30180/flagcicd-backend/metrics/'
+          backend_url: 'http://flagcicd-inner.flagos.net:8000/metrics/'
+          user_id: '000000000000000000'
           report_path: 'coverage-${{ inputs.platform }}.json'
           fail_on_error: 'false'

.github/workflows/_unit_test.yml

@@ -16,6 +16,7 @@ In theory, vllm-plugin-FL can support all models available in vLLM, as long as n
 | MiniCPM-o 4.5 | Supported | [example](./examples/minicpm/) |
 | GLM-5 | Supported | [example](./examples/glm_5_offline_inference.py) |
 | Qwen3.5-35B-A3B | Supported | [example](./examples/glm_5_offline_inference.py)  |
+| BAAI/bge-m3 | Supported | [implementation](./vllm_fl/models/bge_m3.py) |
 
 ### Supported Chips
 

README.md

@@ -6,7 +6,7 @@ Evaluation toolkit for large language models with LM Eval and vLLM Benchmark.
 
 ### 1. Dependencies
 
-* **[lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)** 
+* **[lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)**
     ```bash
     git clone https://github.com/EleutherAI/lm-evaluation-harness.git
     cd lm-evaluation-harness
@@ -39,6 +39,10 @@ cd benchmarks/flagos_eval
 
 # Performance Benchmark
 ./run_benchmark.sh /path/to/model/
+
+# Mixed-Length Benchmark (Vendor Acceptance)
+./run_mixed_benchmark.sh /path/to/model/         # default seed=42
+./run_mixed_benchmark.sh /path/to/model/ 123      # custom seed
 ```
 
 ## Evaluation Tasks
@@ -65,16 +69,41 @@ Output Files:
 - `bench_summary.csv` - Summary
 - `bench_results/*.json` - Detailed results
 
+### Mixed-Length Benchmark
+
+Runs throughput tests with mixed input lengths in shuffled order for vendor performance acceptance testing. Uses a seed to ensure reproducible shuffle results.
+
+Configuration (1610 samples total):
+
+| Input/Output | Samples |
+|--------------|---------|
+| 1k/1k       | 500     |
+| 2k/1k       | 300     |
+| 4k/1k       | 300     |
+| 6k/1k       | 300     |
+| 12k/1k      | 200     |
+| 32k/1k      | 10      |
+
+Output Files:
+- `mixed_bench_summary.csv` - Summary
+- `mixed_bench_summary.json` - Summary (JSON)
+- `mixed_bench_results/*.json` - Detailed results
+
 ## Project Structure
 
 ```
 flagos_eval/
-├── run_eval.sh                  # LM evaluation script
-├── run_benchmark.sh             # Performance benchmark script
-├── collect_eval_results.py           # LM evaluation result collector
+├── run_eval.sh                      # LM evaluation script
+├── run_benchmark.sh                 # Performance benchmark script
+├── run_mixed_benchmark.sh           # Mixed-length benchmark script
+├── collect_eval_results.py          # LM evaluation result collector
 ├── collect_benchmark_results.py     # Performance benchmark result collector
-├── output/                      # LM evaluation results
-├── bench_results/               # Performance benchmark results
-├── results_summary.csv          # LM evaluation summary
-└── bench_summary.csv            # Performance benchmark summary
+├── collect_mixed_benchmark_results.py # Mixed-length result collector
+├── output/                          # LM evaluation results
+├── bench_results/                   # Performance benchmark results
+├── mixed_bench_results/             # Mixed-length benchmark results
+├── results_summary.csv              # LM evaluation summary
+├── bench_summary.csv                # Performance benchmark summary
+├── mixed_bench_summary.csv          # Mixed-length benchmark summary
+└── mixed_bench_summary.json         # Mixed-length benchmark summary (JSON)
 ```