Skip to content

Commit 1cc3947

Browse files
authored
add Ascend guide (#75)
1 parent eb0edbf commit 1cc3947

1 file changed

Lines changed: 32 additions & 12 deletions

File tree

README.md

Lines changed: 32 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -30,18 +30,18 @@ In theory, vllm-plugin-FL can support all models available in vLLM, as long as n
3030

3131
### Setup
3232

33-
0. Install vllm from the official [v0.13.0](https://github.com/vllm-project/vllm/tree/v0.13.0) (optional if the correct version is installed) or from the fork [vllm-FL](https://github.com/flagos-ai/vllm-FL).
33+
1. Install vllm from the official [v0.13.0](https://github.com/vllm-project/vllm/tree/v0.13.0) (optional if the correct version is installed) or from the fork [vllm-FL](https://github.com/flagos-ai/vllm-FL).
3434

3535

36-
1. Install vllm-plugin-FL
36+
2. Install vllm-plugin-FL
3737

38-
1.1 Clone the repository:
38+
2.1 Clone the repository:
3939

4040
```sh
4141
git clone https://github.com/flagos-ai/vllm-plugin-FL
4242
```
4343

44-
1.2 install
44+
2.2 install
4545
```sh
4646
cd vllm-plugin-FL
4747
pip install -r requirements.txt
@@ -50,15 +50,15 @@ In theory, vllm-plugin-FL can support all models available in vLLM, as long as n
5050
pip install --no-build-isolation -e .
5151
```
5252

53-
2. Install [FlagGems](https://github.com/flagos-ai/FlagGems/blob/master/docs/getting-started.md#quick-installation)
53+
3. Install [FlagGems](https://github.com/flagos-ai/FlagGems/blob/master/docs/getting-started.md#quick-installation)
5454

55-
2.1 Install Build Dependencies
55+
3.1 Install Build Dependencies
5656

5757
```sh
5858
pip install -U scikit-build-core==0.11 pybind11 ninja cmake
5959
```
6060

61-
2.2 Installation FlagGems
61+
3.2 Installation FlagGems
6262

6363
```sh
6464
git clone https://github.com/flagos-ai/FlagGems
@@ -68,27 +68,27 @@ In theory, vllm-plugin-FL can support all models available in vLLM, as long as n
6868
pip install --no-build-isolation -e .
6969
```
7070

71-
3. Install [FlagCX](https://github.com/flagos-ai/FlagCX/blob/main/docs/getting_started.md#build-and-installation)
71+
4. (Optional) Install [FlagCX](https://github.com/flagos-ai/FlagCX/blob/main/docs/getting_started.md#build-and-installation)
7272

73-
3.1 Clone the repository:
73+
4.1 Clone the repository:
7474
```sh
7575
git clone https://github.com/flagos-ai/FlagCX.git
7676
cd FlagCX
7777
git checkout -b v0.9.0
7878
git submodule update --init --recursive
7979
```
8080

81-
3.2 Build the library with different flags targeting to different platforms:
81+
4.2 Build the library with different flags targeting to different platforms:
8282
```sh
8383
make USE_NVIDIA=1
8484
```
8585

86-
3.3 Set environment
86+
4.3 Set environment
8787
```sh
8888
export FLAGCX_PATH="$PWD"
8989
```
9090

91-
3.4 Installation FlagCX
91+
4.4 Installation FlagCX
9292
```sh
9393
cd plugin/torch/
9494
FLAGCX_ADAPTOR=[xxx] pip install . --no-build-isolation
@@ -100,6 +100,26 @@ In theory, vllm-plugin-FL can support all models available in vLLM, as long as n
100100

101101
If there are multiple plugins in the current environment, you can specify use vllm-plugin-fl via VLLM_PLUGINS='fl'.
102102

103+
### Additional Steps for Ascend
104+
105+
1. Install [FlagTree](https://resource.flagos.net)
106+
107+
```sh
108+
RES="--index-url=https://resource.flagos.net/repository/flagos-pypi-hosted/simple --trusted-host=https://resource.flagos.net"
109+
python3 -m pip install flagtree==0.4.0+ascend3.2 $RES
110+
```
111+
112+
2. Set required environment variable
113+
114+
```sh
115+
export TRITON_ALL_BLOCKS_PARALLEL=1
116+
```
117+
118+
3. Enable eager execution
119+
120+
Ascend requires eager execution. Add `enforce_eager=True` to the `LLM` constructor or pass `--enforce-eager` on the command line.
121+
122+
103123
### Run a Task
104124

105125
#### Offline Batched Inference

0 commit comments

Comments
 (0)