|
| 1 | +# HRX System |
| 2 | + |
| 3 | +> [!CAUTION] |
| 4 | +> This is early-access runtime infrastructure. Expect API, packaging, and |
| 5 | +> deployment details to move while the fork is brought up. This is not an |
| 6 | +> official component within the ROCm stack. |
| 7 | +
|
| 8 | +The HRX System is a collection of minimal runtime components providing an |
| 9 | +alternative implementation of HIP as it presently ships within ROCm. It |
| 10 | +provides a common substrate for low latency, high performance integration |
| 11 | +with AMD's GPU, NPU, and CPU products. |
| 12 | + |
| 13 | +## What Is Included |
| 14 | + |
| 15 | +- `runtime/`: core runtime components |
| 16 | +- `libhrx/`: the public HRX C ABI, C++ helpers, HIP compatibility binding, |
| 17 | + streaming support, CTS, and debug passthrough tools. |
| 18 | +- `build_tools/`: CMake/Bazel translation helpers, CI packaging scripts, test |
| 19 | + install support, sanitizer wiring, and dependency discovery. |
| 20 | + |
| 21 | +The public install exports `libhrx.so`, `hrx-info`, public HRX headers for the |
| 22 | +native low-latency API, CMake package files, and the HRX HIP compatibility |
| 23 | +layer as `lib/libamdhip64.so`. |
| 24 | + |
| 25 | +## Building |
| 26 | + |
| 27 | +Use a ROCm installation or unpacked ROCm build environment that provides HSA, |
| 28 | +AQL profile headers, and a ROCm LLVM toolchain. The CI path uses ROCm clang and |
| 29 | +forces libraries into `lib`. |
| 30 | + |
| 31 | +```bash |
| 32 | +cmake -S . -B build/hrx-system -GNinja \ |
| 33 | + -DCMAKE_PREFIX_PATH=/opt/rocm \ |
| 34 | + -DCMAKE_INSTALL_LIBDIR=lib \ |
| 35 | + -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang \ |
| 36 | + -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ \ |
| 37 | + -DCMAKE_ASM_COMPILER=/opt/rocm/llvm/bin/clang \ |
| 38 | + -DCMAKE_AR=/opt/rocm/llvm/bin/llvm-ar \ |
| 39 | + -DCMAKE_RANLIB=/opt/rocm/llvm/bin/llvm-ranlib \ |
| 40 | + -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" \ |
| 41 | + -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=lld" \ |
| 42 | + -DCMAKE_MODULE_LINKER_FLAGS="-fuse-ld=lld" \ |
| 43 | + -DCMAKE_BUILD_TYPE=RelWithDebInfo \ |
| 44 | + -DIREE_ROCM_TEST_TARGET_CHIP=gfx942 |
| 45 | + |
| 46 | +cmake --build build/hrx-system |
| 47 | +cmake --install build/hrx-system --prefix build/hrx-install \ |
| 48 | + --component HrxPublicDist |
| 49 | +cmake --install build/hrx-system --prefix build/hrx-tests \ |
| 50 | + --component HrxTestsDist |
| 51 | +``` |
| 52 | + |
| 53 | +Useful options: |
| 54 | + |
| 55 | +| Option | Default | Purpose | |
| 56 | +|--------|---------|---------| |
| 57 | +| `HRX_HERMETIC_BUILD` | OFF | Error instead of using FetchContent fallbacks. | |
| 58 | +| `LIBHRX_BUILD` | ON | Build libhrx and compatibility targets. | |
| 59 | +| `LIBHRX_BUILD_CTS` | `${IREE_BUILD_TESTS}` | Build libhrx CTS binaries. | |
| 60 | +| `LIBHRX_BUILD_PASSTHROUGH` | ON | Build HIP passthrough/interception tools. | |
| 61 | +| `IREE_BUILD_TESTS` | ON | Build IREE runtime tests and CTS targets. | |
| 62 | +| `HRX_INSTALL_TESTS` | `${IREE_BUILD_TESTS}` | Install a relocatable CTest tree. | |
| 63 | +| `IREE_HAL_DRIVER_AMDGPU` | ON | Build the AMDGPU runtime HAL driver. | |
| 64 | +| `IREE_ROCM_TEST_TARGET_CHIP` | empty | Target chip for ROCm tests that compile device code. | |
| 65 | + |
| 66 | +`HRX_HERMETIC_BUILD=ON` is intended for release and distro-style builds where |
| 67 | +all dependencies must be supplied by package discovery. |
| 68 | + |
| 69 | +## Running Tests |
| 70 | + |
| 71 | +The installed test tree is relocatable and can run outside the build directory: |
| 72 | + |
| 73 | +```bash |
| 74 | +ctest --test-dir build/hrx-tests/share/hrx-system/tests \ |
| 75 | + --output-on-failure --parallel "$(nproc)" |
| 76 | +``` |
| 77 | + |
| 78 | +Set `HRX_TEST_TMPDIR` to put per-test temporary directories somewhere writable: |
| 79 | + |
| 80 | +```bash |
| 81 | +HRX_TEST_TMPDIR=/tmp/hrx-system-tests \ |
| 82 | + ctest --test-dir build/hrx-tests/share/hrx-system/tests \ |
| 83 | + --output-on-failure --parallel "$(nproc)" |
| 84 | +``` |
| 85 | + |
| 86 | +GPU tests are labeled with `requires-gpu-*`. CPU-only runners should exclude |
| 87 | +them: |
| 88 | + |
| 89 | +```bash |
| 90 | +ctest --test-dir build/hrx-tests/share/hrx-system/tests \ |
| 91 | + --output-on-failure -LE requires-gpu- |
| 92 | +``` |
| 93 | + |
| 94 | +## Release Artifacts |
| 95 | + |
| 96 | +Tagged releases publish one tarball per artifact, plus manifests and environment |
| 97 | +helper scripts where useful: |
| 98 | + |
| 99 | +| Artifact | Contents | |
| 100 | +|----------|----------| |
| 101 | +| `hrx-public-linux-x86_64` | Public HRX install: `libhrx.so`, `libamdhip64.so`, headers, CMake package files, and tools. | |
| 102 | +| `hrx-public-deps-linux-x86_64` | Runtime ROCm dependency subset needed with the public install, including HSA and AQL profile libraries. | |
| 103 | +| `hrx-tests-linux-x86_64` | Installed CTest tree, unit tests, CTS binaries, test data, and benchmarks. | |
| 104 | +| `hrx-rocm-buildenv-linux-x86_64` | ROCm build environment used by CI, including ROCm LLVM. | |
| 105 | + |
| 106 | +To recreate the runtime overlay used by CI, extract the public package and the |
| 107 | +public dependency package into the same root: |
| 108 | + |
| 109 | +```bash |
| 110 | +mkdir -p /tmp/hrx-root |
| 111 | +cp hrx-public-linux-x86_64-env.sh /tmp/hrx-root/ |
| 112 | +tar --use-compress-program=unzstd -xf hrx-public-linux-x86_64-<tag>.tar.zst \ |
| 113 | + -C /tmp/hrx-root |
| 114 | +tar --use-compress-program=unzstd -xf hrx-public-deps-linux-x86_64-<tag>.tar.zst \ |
| 115 | + -C /tmp/hrx-root |
| 116 | +source /tmp/hrx-root/hrx-public-linux-x86_64-env.sh |
| 117 | +``` |
| 118 | + |
| 119 | +Extract `hrx-tests-linux-x86_64-<tag>.tar.zst` into a separate tests root and |
| 120 | +run its installed CTest tree against the composed runtime environment. |
| 121 | + |
| 122 | +## Running HIP Applications |
| 123 | + |
| 124 | +The public artifact contains an HRX-provided `lib/libamdhip64.so`. Preload that |
| 125 | +library to route a HIP application through HRX without relinking the |
| 126 | +application: |
| 127 | + |
| 128 | +```bash |
| 129 | +export HRX_ROOT=/tmp/hrx-root |
| 130 | +export LD_LIBRARY_PATH="${HRX_ROOT}/lib:${HRX_ROOT}/lib/rocm_sysdeps/lib:${LD_LIBRARY_PATH:-}" |
| 131 | +export HRX_GPU_DRIVER=amdgpu |
| 132 | + |
| 133 | +LD_PRELOAD="${HRX_ROOT}/lib/libamdhip64.so" ./my_hip_application |
| 134 | +``` |
| 135 | + |
| 136 | +This path is for running HIP-based applications on the HRX compatibility layer. |
| 137 | +The passthrough/debug libraries under `libhrx/src/passthrough/` are separate |
| 138 | +developer tools for comparing behavior against the real ROCm HIP runtime. |
| 139 | + |
| 140 | +## More Documentation |
| 141 | + |
| 142 | +- `docs/testing/installed_tests.md`: installed test tree details. |
| 143 | +- `libhrx/src/passthrough/README.md`: HIP passthrough and tracing tools. |
0 commit comments