candle-kernels build_lib() fails on Windows MSVC + CUDA 13.3 (MoE static library)

[freezed-corpse-143]

Describe the bug

candle-kernels custom build script (cudaforge::build_lib()) fails during compilation of the MoE static library on Windows with MSVC + CUDA 13.3. The PTX build (build_ptx()) succeeds, but the subsequent static library build for MoE/quant kernels exits with code 1 and no detailed error message.

All other CUDA crates (mistralrs-quant, mistralrs-paged-attn, mistralrs-core) compile successfully.

Reproduction

1. Clone the latest master:

git clone https://github.com/EricLBuehler/mistral.rs.git
cd mistral.rs

2. Load MSVC environment and set up CUDA paths:

& "path\to\VsDevCmd.ps1"
$env:CUDA_TOOLKIT_PATH = "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.3"

3. Build:

cargo build --release --package mistralrs-cli --features "cuda cudnn nccl code-execution"

Or to isolate the failing crate:

cargo build --release --package mistralrs-core --features "cuda cudnn"

Error output

error: failed to run custom build command for `candle-kernels v0.11.0`

Caused by:
  process didn't exit successfully: `...\build-script-build` (exit code: 1)
  --- stdout
  cargo::rerun-if-changed=build.rs
  cargo::rerun-if-changed=src/compatibility.cuh
  cargo::rerun-if-changed=src/cuda_utils.cuh
  cargo::rerun-if-changed=src/binary_op_macros.cuh
  cargo:rustc-env=CUDA_INCLUDE_DIR=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.3\include
  cargo:rerun-if-changed=src\affine.cu
  cargo:rerun-if-changed=src\binary.cu
  cargo:rerun-if-changed=src\cast.cu
  cargo:rerun-if-changed=src\conv.cu
  cargo:rerun-if-changed=src\fill.cu
  cargo:rerun-if-changed=src\indexing.cu
  cargo:rerun-if-changed=src\quantized.cu
  cargo:rerun-if-changed=src\reduce.cu
  cargo:rerun-if-changed=src\sort.cu
  cargo:rerun-if-changed=src\ternary.cu
  cargo:rerun-if-changed=src\unary.cu
  cargo:rerun-if-env-changed=CUDA_COMPUTE_CAP
  cargo:rerun-if-env-changed=NVCC_CCBIN
  sort.cu
  affine.cu
  ternary.cu
  ...
  quantized.cu

The build.rs (from candle-kernels pinned at rev=78e1d85) has two stages:

1. Stage 1: build_ptx() — scans src/*.cu, compiles for PTX → succeeds (all .cu files printed)
2. Stage 2: build_lib("libmoe.a") — compiles MoE kernels as a static library → fails with exit code 1

The failing build_lib() call in candle-kernels/build.rs compiles these source files:

• src/moe/moe_gguf.cu
• src/moe/moe_wmma.cu
• src/moe/moe_wmma_gguf.cu
• src/mmvq_gguf.cu
• src/mmq_gguf/mmq_*.cu (6 files)

The error is propagated via ? in the build script, so cudaforge's inner nvcc error message is not displayed. This is likely an nvlink or CRT mismatch issue specific to MSVC + CUDA 13.3.

Environment

OS: Windows 11 Pro (10.0.26200)
CPU: AMD Ryzen 9 5900HX (x86_64)
GPU: NVIDIA GeForce RTX 3080 Laptop GPU (compute 8.6, sm86)
RAM: 32 GB

Rust toolchain: stable-x86_64-pc-windows-msvc
rustc MSRV: 1.94

MSVC: Visual Studio 2026 BuildTools
cl.exe: 14.51.36231

CUDA Toolkit: 13.3.73
nvcc: release 13.3, V13.3.73
NVIDIA Driver: 610.62 (supports CUDA UMD 13.3)

cuDNN: 9.24.0
LLVM/Clang: 22.1.8 (for bindgen)

Cargo.toml workspace version: 0.8.23
candle pinned rev: 78e1d85 (from Cargo.toml)

Additional context

• Building without --features cuda (CPU-only) works fine.
• mistralrs-quant, mistralrs-paged-attn, mistralrs-core all compile their CUDA kernels successfully.
• The build_lib() output file on MSVC would be out_dir\libmoe.lib rather than libmoe.a, but this is already handled in the build script for MSVC targets.
• One related workaround: a user on candle issue #3166 reported that compiling PTX manually via direct nvcc invocation (bypassing bindgen_cuda) resolved a similar issue, though that was a different underlying cause (CUDA path detection).
• Related mistral.rs issue mistralrs-quant v0.8.23 fails to build with Cutlass 7d49e6c7e2f8896c #2292 (CUTLASS pin on CUDA 13.3) was already fixed on master.