Skip to content

feat(install): GPU auto-detection + custom torch index override - #137

Open
Pfannkuchensack wants to merge 5 commits into
invoke-ai:mainfrom
Pfannkuchensack:feat/custom-torch-index-and-gpu-detection
Open

feat(install): GPU auto-detection + custom torch index override#137
Pfannkuchensack wants to merge 5 commits into
invoke-ai:mainfrom
Pfannkuchensack:feat/custom-torch-index-and-gpu-detection

Conversation

@Pfannkuchensack

@Pfannkuchensack Pfannkuchensack commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Custom Torch index Override

Re-implements the custom PyTorch index feature (previously #133) on top of the bootstrap install architecture (#135), and adds hardware auto-detection with a confirmation step in the install flow.

Custom torch index:

  • Add a per-install "Custom PyTorch index" field to the Configure step (validated http(s) URL), threaded through the start-install IPC. Replaces the previous global setting, which sidesteps the stale-override problem.
  • Legacy installs (<6.14): the custom URL replaces the pinned torch index.
  • Bootstrap installs (>=6.14): the torch-family packages are skipped during uv sync (--no-install-package) and installed once from the custom index afterwards, so torch is not downloaded twice. Package names and versions are parsed from the release uv.lock (getTorchPackagesFromLock), scoped to the selected torch platform, with local version tags stripped so ==<version> matches the equivalent build on a different index.
  • Surface the active override in the Review step.

GPU auto-detection from here

  • Port the hardware probe to src/main/gpu-detection.ts (async execFile so it never blocks the main process), exposed via a new util:detect-gpu IPC.
  • The Configure step auto-detects on entry and asks the user to confirm; on "no" or detection failure the manual picker is shown, and auto-detect stays reachable from there. CUDA always prompts for the Nvidia generation, which cannot be auto-detected. The confirmation sub-view is kept in the store so it survives navigating away from and back to the step.

Tested on

  • Linux CPU
  • Linux rcom
  • Linux Cuda
  • Windows CPU
  • Windows Rcom
  • Windows Cuda
  • Apple

Pfannkuchensack and others added 2 commits July 5, 2026 06:30
Re-implements the custom PyTorch index feature (previously invoke-ai#133) on top of
the bootstrap install architecture (invoke-ai#135), and adds hardware auto-detection
with a confirmation step in the install flow.

Custom torch index:
- Add a per-install "Custom PyTorch index" field to the Configure step
  (validated http(s) URL), threaded through the start-install IPC. Replaces
  the previous global setting, which sidesteps the stale-override problem.
- Legacy installs (<6.14): the custom URL replaces the pinned torch index.
- Bootstrap installs (>=6.14): the torch-family packages are skipped during
  `uv sync` (`--no-install-package`) and installed once from the custom
  index afterwards, so torch is not downloaded twice. Package names and
  versions are parsed from the release uv.lock (getTorchPackagesFromLock),
  scoped to the selected torch platform, with local version tags stripped so
  `==<version>` matches the equivalent build on a different index.
- Surface the active override in the Review step.

GPU auto-detection:
- Port the hardware probe to src/main/gpu-detection.ts (async execFile so it
  never blocks the main process), exposed via a new util:detect-gpu IPC.
- The Configure step auto-detects on entry and asks the user to confirm; on
  "no" or detection failure the manual picker is shown, and auto-detect stays
  reachable from there. CUDA always prompts for the Nvidia generation, which
  cannot be auto-detected. The confirmation sub-view is kept in the store so
  it survives navigating away from and back to the step.

@lstein lstein left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. A Claude Code review flagged a few issues, all easily addressed:

  • The lock parser is correct against reality. I ran getTorchPackagesFromLock against InvokeAI's real (~899 KB) uv.lock: it returns exactly torch 2.7.1 / torchvision 0.22.1 for cuda and cpu, and torch 2.10.0 / torchvision 0.25.0 / triton-rocm 3.6.0 for rocm. The ^-anchored regex correctly ignores the many indented source = {…} inline tables inside other packages' dependency lists, which the real lock is full of.
  • Override placement is correct: the torchIndexOverride ?? pins.torchIndexUrl[…] fallback sits in the legacy branch only; the bootstrap invokeai package install stays --no-deps with no index flag.
  • All 8 tests pass with a clean type-check, and they cover the right traps (nested dependency tables, pypi-vs-index dedup, per-platform selection, the empty case).
  • Stripping the local version tag so ==2.7.1 matches 2.7.1+cu126 on another index is PEP 440-correct, and the skip-during-sync + single-download design is sound — including self-healing on a later update without the override (the local tags differ, so uv sync restores the lock's builds).

Two inline comments cover the substantive issues (a silent-fallback flag choice, and an xformers ABI caveat). Beyond those, some smaller notes:

Windows AMD detection (low, UX): all the AMD probes (amd-smi, rocm-smi, rocminfo, sysfs/KFD) are Linux-only, so a Windows machine with a discrete Radeon is told "we detected no dedicated GPU (CPU only)". The outcome is defensible — there's no ROCm-on-Windows path — but the message is factually wrong on that hardware, and those are exactly the users the custom-index field targets. A cheap WMI/CIM display-controller probe would let the message say "AMD GPU detected, but ROCm isn't supported on Windows — using CPU."

macOS confirmation friction (low): with the OS-based default gone, every install/update steps through GPU confirmation. That's arguably an improvement (click-through wrong defaults are how AMD users got broken installs), but on macOS "We detected a Mac GPU (Metal / MPS). Is this correct?" has only one sane answer — consider auto-confirming on darwin.

Nits:

  • detectGpu reports confidence: 'high' for the CPU fallback whose reason is "no evidence found" — semantically that's low/none; the 'low' variant is never produced; vendor is stringly-typed and unused by the UI.
  • The main process trusts the renderer's http(s) validation — a one-line re-check in startInstall would be cheap defense-in-depth. Relatedly, credentials embedded in an index URL get echoed into the install log.
  • Worst-case detection latency is ~9s (the ROCm chain runs up to three 3s-timeout probes sequentially); running them concurrently would tighten the spinner time.
  • isCustomTorchIndexUrlInvalid exported from a component file — a shared util might be a more natural home.

One manual test worth adding to your matrix before merge, given the inline --index comment: point the custom index at an index that lacks the locked version and confirm the failure is loud rather than silent.

Overall: solid, well-commented work that fits the codebase conventions. Happy to approve once the --index flag question is addressed and there's a decision on the xformers warning. Thanks again!

🤖 Review assisted by Claude Code

Comment thread src/main/install-manager.ts Outdated
Pfannkuchensack and others added 3 commits July 11, 2026 21:07
Make the custom PyTorch index override fail loudly and safely, and tighten
GPU auto-detection based on PR invoke-ai#137 review.

Custom torch index:
- Bootstrap installs: install the torch-family packages with `--index-url`
  instead of `--index`, so the custom index is the sole index. `--index`
  only prepends, letting uv silently fall back to the default PyPI wheel
  when the custom index lacks the pinned (tag-stripped) version - the exact
  wrong-backend failure this feature prevents. The legacy path keeps
  `--index=` to mirror the pre-existing pins behavior.
- Warn when a custom index is combined with the xformers extra (nvidia<30xx):
  xformers is built against Invoke's default CUDA torch, so a mismatched
  custom CUDA build can cause import errors or crashes. Also noted in the
  field tooltip.
- Re-validate the override as an http(s) URL in the main process instead of
  trusting the renderer, and redact any embedded credentials from the
  install log and surfaced command lines.
- Move URL helpers to shared/url.ts so renderer and main share one rule.

GPU detection:
- Fix the CPU fallback reporting `confidence: 'high'` when no evidence was
  found (now `'none'`); type `vendor` as a GpuVendor union instead of string.
- Run the ROCm tool probes concurrently, cutting worst-case detection
  latency from ~9s to ~3s.
- Detect a discrete AMD GPU on Windows and tell the user ROCm isn't
  supported there (CPU will be used) instead of "no dedicated GPU".
- Auto-confirm the detected Metal backend on macOS to skip a redundant prompt.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants