how to install the NVIDIA’s deep learning stack on Windows: CUDA toolkit + cuDNN + TensorRT RTX + TensorRT,
also various attempts to compile deep learning libraries with cuda
only for Windows 10/11 + at least RTX 20xx GPU
all comands here are for cmd, if u prefer pwsh must change to adapt
some values are censored with black box ███ so u have to fill in with your use case
1st of all: 👉👉👉 install CUDA toolkit + cuDNN + TensorRT RTX + TensorRT 👈👈👈
my attempts to build various deep learning libraries:
- build onnxruntime ✅
- build whisper.cpp with cuBLAS ✅
build CTranslate2 to use faster-whisper❌build triton to use xformers❌try out TensorRT-LLM❌- build OpenCV with CUDA ✅
build Torch-TensorRT❌- build llama.cpp with cuBLAS ✅
- build Stan with OpenCL ✅
all above guides are under CC-BY-4.0
more complicated tweaking with TensorRT (under GPLv3)
- build Real-ESRGAN with TensorRT on Windows: https://github.com/phineas-pta/RealESRGAN-trt-win ✅
- run Stable Diffusion XL with TensorRT natively on Windows: https://github.com/phineas-pta/SDXL-trt-win ✅