Add PyTorch API tests in CICD #8
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Torch API Tests in Container | |
| on: | |
| push: | |
| branches: | |
| - main | |
| pull_request: | |
| branches: | |
| - main | |
| jobs: | |
| torch-api-test: | |
| runs-on: self-hosted | |
| container: | |
| image: localhost:5000/flagscale:cuda12.4.1-cudnn9.5.0-python3.12-torch2.6.0-time2505241715 | |
| options: --gpus all --privileged --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 | |
| steps: | |
| - name: Checkout Code | |
| uses: actions/checkout@v4 | |
| with: | |
| repository: ${{ github.event.pull_request.head.repo.full_name }} | |
| ref: ${{ github.event.pull_request.head.ref }} | |
| ssh-strict: true | |
| ssh-user: git | |
| persist-credentials: true | |
| clean: true | |
| sparse-checkout-cone-mode: true | |
| fetch-tags: false | |
| show-progress: true | |
| lfs: false | |
| submodules: false | |
| set-safe-directory: true | |
| - name: Set up Install Dependencies | |
| run: | | |
| /root/miniconda3/bin/conda activate flagscale-train | |
| apt update -y | |
| apt-get install -y git rdma-core ibverbs-utils perftest infiniband-diags libibverbs-dev librdmacm-dev | |
| cd /__w/FlagCX/FlagCX | |
| git config --global --add safe.directory /__w/FlagCX/FlagCX | |
| git status | |
| pip install setuptools==77.0.1 pre-commit -i https://pypi.tuna.tsinghua.edu.cn/simple | |
| pre-commit install | |
| - name: Run Code Format Check with pre-commit | |
| run: | | |
| cd /__w/FlagCX/FlagCX | |
| apt update -y | |
| apt-get install clang-format -y | |
| git fetch --all | |
| from_ref=${GITHUB_HEAD_REF} | |
| to_ref=${GITHUB_BASE_REF} | |
| echo "From branch: $from_ref, To branch: $to_ref" | |
| pre-commit run --from-ref "origin/$from_ref" --to-ref "origin/$to_ref" | |
| continue-on-error: false | |
| - name: Run `make` to build the project | |
| run: | | |
| cd /__w/FlagCX/FlagCX | |
| export MPI_HOME=/usr/local/mpi | |
| make USE_NVIDIA=1 | |
| - name: Enter plugin/torch and install FlagCX PyTorch Plugin | |
| run: | | |
| cd /__w/FlagCX/FlagCX/plugin/torch | |
| python setup.py develop --adaptor nvidia | |
| - name: Run PyTorch API tests | |
| run: | | |
| cd /__w/FlagCX/FlagCX/test/script/ | |
| bash torch_api_test.sh |