|
| 1 | +Continuous Integration (CI) tests |
| 2 | +================================ |
| 3 | + |
| 4 | +This document describes the GitHub Actions continuous-integration setup used to run the Idefix |
| 5 | +test-suite. The CI is implemented by two workflows checked in .github/workflows: |
| 6 | + |
| 7 | +- .github/workflows/idefix-ci.yml |
| 8 | +- .github/workflows/idefix-ci-jobs.yml |
| 9 | + |
| 10 | +Overview |
| 11 | +-------- |
| 12 | + |
| 13 | +The CI is split in two layers: |
| 14 | + |
| 15 | +- A top-level workflow (.github/workflows/idefix-ci.yml) that: |
| 16 | + |
| 17 | + - runs a Linter job (pre-commit) on push / PR / manual dispatch, |
| 18 | + - then calls a reusable workflow for different compiler/backends (intel, gcc, cuda) |
| 19 | + providing two inputs: TESTME_OPTIONS and IDEFIX_COMPILER. |
| 20 | + |
| 21 | +- A reusable workflow (.github/workflows/idefix-ci-jobs.yml) that: |
| 22 | + |
| 23 | + - defines the actual test jobs grouped by physics domain (ShocksHydro, ParabolicHydro, |
| 24 | + ShocksMHD, ParabolicMHD, Fargo, ShearingBox, SelfGravity, Planet, Dust, Braginskii, |
| 25 | + Examples, Utils), |
| 26 | + - runs test scripts on self-hosted runners, |
| 27 | + - expects the repository to be checked out with submodules, |
| 28 | + - invokes the repository-provided CI helper scripts to configure / build / run tests. |
| 29 | + |
| 30 | +Key configuration points |
| 31 | +------------------------ |
| 32 | + |
| 33 | +- Inputs passed from the top-level workflow: |
| 34 | + |
| 35 | + - TESTME_OPTIONS (string): flags forwarded to the per-test runner (examples: -cuda, -Werror, |
| 36 | + -intel, -all). |
| 37 | + - IDEFIX_COMPILER (string): which compiler the tests should use (e.g. icc, gcc, nvcc). |
| 38 | + |
| 39 | +- Environment variables set by the reusable workflow: |
| 40 | + |
| 41 | + - IDEFIX_COMPILER, TESTME_OPTIONS, PYTHONPATH, IDEFIX_DIR |
| 42 | + |
| 43 | +- Linter job: |
| 44 | + |
| 45 | + - Runs only when repository is the main project (not arbitrary forks). |
| 46 | + - Uses actions/setup-python and runs pre-commit (pre-commit/action@v3 and pre-commit-ci/lite). |
| 47 | + - Prevents regressions in style and common mistakes before running heavy test jobs. |
| 48 | + |
| 49 | +- Test execution: |
| 50 | + |
| 51 | + - All test jobs call the repository script scripts/ci/run-tests with a test directory |
| 52 | + and the TESTME_OPTIONS flags. Example invocation (from the workflows): |
| 53 | + scripts/ci/run-tests $IDEFIX_DIR/test/HD/sod -all $TESTME_OPTIONS |
| 54 | + |
| 55 | + - The reusable workflow is written to execute many test directories in separate job steps, |
| 56 | + so each physics group is kept logically separated in CI logs. |
| 57 | + |
| 58 | +Runners and prerequisites |
| 59 | +------------------------- |
| 60 | + |
| 61 | +- The heavy numerical tests run on self-hosted runners (see runs-on: self-hosted). |
| 62 | + The CI assumes appropriate hardware and dependencies are available on those runners |
| 63 | + (compilers, MPI, GPUs when CUDA/HIP flags are used, required system libraries). |
| 64 | + |
| 65 | +- The workflows check out the repository and its submodules. Submodules must be available |
| 66 | + on the CI machines. |
| 67 | + |
| 68 | +How tests are driven (testme scripts) |
| 69 | +------------------------------------- |
| 70 | + |
| 71 | +Each test directory contains a small Python "testMe" driver that uses the helper Python |
| 72 | +class documented in the repository: |
| 73 | + |
| 74 | +- See the test helper documentation: :doc:`idfxTest <testing/idfxTest>` |
| 75 | + |
| 76 | +That helper (idfxTest) is responsible for: |
| 77 | + |
| 78 | +- parsing TESTME_OPTIONS-like flags (precision, MPI, CUDA, reconstruction, vector potential, etc.), |
| 79 | +- calling configure / compile / run, |
| 80 | +- performing standard python checks and non-regression (RMSE) comparisons against |
| 81 | + reference dumps, |
| 82 | +- optionally creating / updating reference dumps (init mode). |
| 83 | + |
| 84 | +Practical examples |
| 85 | +------------------ |
| 86 | + |
| 87 | +- Example of a CI invocation (triggered by workflows): |
| 88 | + |
| 89 | + - Top-level workflow calls the reusable jobs workflow for each compiler/back-end, e.g. |
| 90 | + TESTME_OPTIONS="-cuda -Werror" IDEFIX_COMPILER=nvcc |
| 91 | + |
| 92 | +- Running tests locally (developer machine) |
| 93 | + - You can mimic what CI does by calling the repository helper script directly. Example: |
| 94 | + scripts/ci/run-tests /path/to/idefix/test/HD/sod -all -mpi -dec 2 2 -reconstruction 3 -single |
| 95 | + |
| 96 | +Notes for maintainers |
| 97 | +--------------------- |
| 98 | + |
| 99 | +- The reusable jobs workflow contains a commented concurrency block for optional cancellation |
| 100 | + of in-flight runs — consider enabling it if you want to auto-cancel redundant CI runs. |
| 101 | +- Because tests are run on self-hosted runners, ensure the pools have the required compilers, |
| 102 | + MPI stacks and GPU drivers for the requested TESTME_OPTIONS. |
| 103 | +- Keep TESTME_OPTIONS in sync with the options understood by the test helper documented in |
| 104 | + :doc:`idfxTest <testing/idfxTest>`. |
| 105 | + |
| 106 | +Relevant files |
| 107 | +-------------- |
| 108 | + |
| 109 | +- Workflow entry point: .github/workflows/idefix-ci.yml |
| 110 | +- Reusable jobs: .github/workflows/idefix-ci-jobs.yml |
| 111 | +- Test helper documentation: :doc:`idfxTest <testing/idfxTest>` |
| 112 | + |
| 113 | +.. toctree:: |
| 114 | + :maxdepth: 2 |
| 115 | + :caption: Contents: |
| 116 | + |
| 117 | + testing/idfxTest.rst |
0 commit comments