Add PyTorch API tests in CICD #17
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Torch API Tests in Container | |
| on: | |
| push: | |
| branches: | |
| - main | |
| pull_request: | |
| branches: | |
| - main | |
| jobs: | |
| torch-api-test: | |
| runs-on: self-hosted | |
| container: | |
| image: localhost:5000/flagscale:cuda12.4.1-cudnn9.5.0-python3.12-torch2.6.0-time2505241715 | |
| options: --gpus all --privileged --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 | |
| steps: | |
| - name: Checkout Code | |
| uses: actions/checkout@v4 | |
| with: | |
| repository: ${{ github.event.pull_request.head.repo.full_name }} | |
| ref: ${{ github.event.pull_request.head.ref }} | |
| ssh-strict: true | |
| ssh-user: git | |
| persist-credentials: true | |
| clean: true | |
| sparse-checkout-cone-mode: true | |
| fetch-tags: false | |
| show-progress: true | |
| lfs: false | |
| submodules: false | |
| set-safe-directory: true | |
| - name: Set up Install Dependencies | |
| run: | | |
| export PATH=/root/miniconda3/bin:$PATH | |
| eval "$(/root/miniconda3/bin/conda shell.bash hook)" && conda activate flagscale-train && conda env list | |
| pip list | grep torch && which python | |
| apt update -y | |
| apt-get install -y git rdma-core ibverbs-utils perftest infiniband-diags libibverbs-dev librdmacm-dev clang-format | |
| cd /__w/FlagCX/FlagCX | |
| git config --global --add safe.directory /__w/FlagCX/FlagCX && git status | |
| pip uninstall -y setuptools && pip install -y setuptools==77.0.1 -i https://pypi.tuna.tsinghua.edu.cn/simple | |
| pip install -y pre-commit -i https://pypi.tuna.tsinghua.edu.cn/simple && pre-commit install | |
| - name: Run Code Format Check with pre-commit | |
| run: | | |
| cd /__w/FlagCX/FlagCX | |
| git fetch --all | |
| from_ref=${GITHUB_HEAD_REF} | |
| to_ref=${GITHUB_BASE_REF} | |
| echo "From branch: $from_ref, To branch: $to_ref" | |
| pre-commit run --from-ref "origin/$from_ref" --to-ref "origin/$to_ref" | |
| continue-on-error: false | |
| - name: Run `make` to build the project | |
| run: | | |
| cd /__w/FlagCX/FlagCX | |
| export MPI_HOME=/usr/local/mpi | |
| make USE_NVIDIA=1 -j$(nproc) | |
| - name: Enter plugin/torch and install FlagCX PyTorch Plugin | |
| run: | | |
| cd /__w/FlagCX/FlagCX/plugin/torch | |
| python setup.py develop --adaptor nvidia | |
| - name: Run PyTorch API tests | |
| run: | | |
| cd /__w/FlagCX/FlagCX/test/script/ | |
| bash torch_api_test.sh |