use Metal's Indirect Command Buffers for true GPU-side multi draw indirect - #9640
use Metal's Indirect Command Buffers for true GPU-side multi draw indirect#9640matthargett wants to merge 30 commits into
Conversation
|
Holy shit this is large. I will take a look but it might be a while. In the meantime,
Also, I must ask, to what extent is the code written by LLMs? What about the PR description? |
Sorry, I did try to keep it to the thinnest meaningful slice that demonstrated e2e uplift in my AR portal on real devices. if you see a natural seam I can split things out on, or I should disabuse myself of the constraint I imposed on myelf that a single PR bringing measurable uplift, just let me know! :D
I didn't deal with multi_draw_indirect_count / multi_draw_indexed_indirect_count yet, as one of the ways to keep the PR's size down. It is intentionally scoped to "just" fixed-count multi_draw_indirect and multi_draw_indexed_indirect. The count-buffer variants have additional semantics and could/should be a separate follow-up.
Mesh shaders are supported on my local M4 MacBook device, and I verified that existing fixed-count mesh MDI tests pass, but this PR doesn't add a Metal ICB mesh-command generation path. I did try to control scope, so count-buffer MDI and draw_index would need to be follow-ons unless it needs to be one atomic PR/commit from your perspective. (side note: I found mesh shaders to be pretty finicky on 26.0 operating systems when using MSL directly, so I'd anticipate even more physical device testing and iteration)
nope. In my notebook when I reviewed wgpu, I wrote that Naga’s MSL backend still rejects that builtin. (if that's changed or I was mistaken, do tell me!) would it be useful for me to add a test/fallback assertion so the ICB path cannot give the wrong impression on what it supports?
the hardest part of this stuff is all the on-device testing (especially with ICBs which can halt/panic a device), and I try to capture all the quirks I ran into so others can avoid the tediousness of some of it on legacy devices that don't receive active update.
I came up with the high-level design for the WASM fantasy console in my notebooks, and MDI was added into that requirements list when I first learned about it at the Khronos meetup at GDC a few years ago. Then I used codex and iterated with it over the course of a few months to push the boundaries of older/weaker Apple GPUs (namely, iPhone XS and Apple Watch SE2/6). I built up quite a local patch stack (which I reviewed with my own eyes at each step along the way) while creating all these demonstrations (WebAudio WASI using WebGPU in WASM worklets), and then directed my attention back to BabylonJS/BabylonNative which my startup's main product was written in. I had tried several times last year to fix bgfx's swapchains so we could ship a native BabylonJS app to the AVP app store, but then realized I just needed to rip off the bandaid and do the work to integrate wgpu-native. I used Codex to drive that work, but going through all their screenshot tests manually and spot checking accuracy was not left to AI. Similarly, with the AR portal integration test, I had to physically pick up iPhones and iPads to scan the room after each build (so ARkit could detect the floor), walk into an out of the portal, note problems, and iterate with Codex. Then, I directed Codex to measure e2e framerate/CPU stats, alongside XCode CPU profiler data (L1/prefetch/branch-prediction misses), and guided it to the optimization result of 60 fps camera feed+3D AR projection on iPhone 12. (Some of those optimizations were on the Babylon.js side, and several of those submitted PRs have been merged/released by the nice folks at Microsoft.) wrt the PR text: I directed Codex to summarize the important hard-won knowledge not represented in the diff, and gave it the bullet outline that it elaborated, but I also edited the PR title and description in the staging PR in my wgpu fork. I then had Claude Code review the fork PR description, commits, etc to look for problems but also ways it could better conform to gfx-rs project PR/issue norms/templates. If you look at my open source contributions across the last ~30 years, I can be very verbose especially when it comes to performance, exploits, and optimization. So yes, I used multiple AI coding tools, but I personally reviewed and guided each step of the way, and did not post a PR in this project until it looked good (and passed CI) in my fork. Sorry for the wall of text, but I wanted to give a nuanced response so you (and whomever else) can understand this wasn't a one-shot AI prompt on a whim: I have things built with wgpu that I'd like to ship, and getting better FPS per milliwatt of power draw means MDI is key. |
|
Ok thank you for the explanations! |
|
Looking at this on my phone so pardon any misunderstandings:
|
|
Also, you should add testing to wgpu's own test suite for every new piece of api surface, if you haven't already. I would also like to see benchmarks that these indirect command buffers do actually speed things up, though I don't doubt it. This is not a requirement though |
Add Metal ICB generation for mesh multi-draw indirect and opt-in count-buffer variants. Wire the Chromium experimental multi-draw indirect API surface for CTS, add wgpu-owned readback tests for normal/indexed count-buffer draws and mesh MDI, and validate draw-count-buffer offset alignment.
okay, based on our discord discussion and running more of the Dawn CTS, I expanded the test coverage and added some fallback code for when the feature isn't supported. Now ppl won't be surprised on Metal, and the accurate exception did traverse to my crash reporter (Sentry) in my integration native app
It should be sort of straightforward, modulo discovering silicon/driver quirks across the test devices.
same as above, should be straightforward. I can submit these PRs in parallel (based on eachother's PR branch), if you want to see more of the e2e all at once without the first PR being giant.
yea, I see that in other repos as well. anyone I've worked with will tell you my biggest weakness is being verbose in my comms. I review everything in my fork before I ever submit work to upstream repos. |
|
For anyone else curious, I talked privately with Matt Hargett and he seems like a real and very experienced person.
Not your obligation. I was just not sure if this approach would have to be redone for either of those to be implemented in the future, but you don't need to implement those right now. |
inner-daemons
left a comment
There was a problem hiding this comment.
The final comment here is my main concern. I am happy to have this but that is a serious problem that needs to be addressed.
Also, I think that you were a little too willing to use environment variables. We usually don't accept environment variables as the only way to control behavior, especially in hal. And these cases are more hacky from what I understand.
Overall, I look forward to having this PR in eventually, and I'm grateful to you for putting in the effort to move towards that.
| #[error("Indirect draw count buffer offset {0:?} is not a multiple of 4")] | ||
| UnalignedIndirectCountBufferOffset(BufferAddress), |
There was a problem hiding this comment.
IMO there should be a limit for the indirect args alignment, rather than a hardcoded value.
There was a problem hiding this comment.
Done — added wgpu_types::INDIRECT_BUFFER_OFFSET_ALIGNMENT and both the indirect-offset and the new count-offset checks use it (in #9679, where the count validation now lives). Since the value is fixed at 4 by the WebGPU spec and no backend requires stricter, I went with a named constant à la COPY_BUFFER_ALIGNMENT rather than a queryable limit — happy to promote it to a real Limits entry if you'd prefer it queryable per adapter.
…aw-indirect # Conflicts: # CHANGELOG.md # deno_webgpu/render_pass.rs
Extract the inline MSL generation kernels into wgpu-hal/src/metal/shaders/ and include_str! them, following the gles/shaders precedent, and document the ICB tuning and primitive-topology constants. Requested in review of gfx-rs#9640.
Document why MULTI_DRAW_INDIRECT_COUNT is exposed past all_webgpu_mask(): it surfaces as Chromium's experimental chromium-experimental-multi-draw-indirect WebGPU feature. Requested in review of gfx-rs#9640.
Move the MULTI_DRAW_INDIRECT_COUNT surface (deno bindings and the chromium-experimental-multi-draw-indirect mapping) out to gfx-rs#9679, which owns the count feature. Requested in review: the env-gated feature exposure didn't belong in this PR.
Rework requested in review of gfx-rs#9640: instead of suspending and resuming the render encoder around each multi-draw, the ICB is allocated and its execution recorded at draw time, and the reset/generate/optimize work is deferred into the same internal pre-pass command buffer that carries indirect-draw validation, via a new default-no-op hal method CommandEncoder::encode_deferred_multi_draws. The queue executes that buffer before the pass, so the pass is never interrupted and no render state needs to be captured or restored. This deletes the suspend/resume machinery, the render-state shadow tracking, the store-action rewriting, the pass-resumability gating, and both environment variables (WGPU_METAL_FORCE_ICB_MDI, WGPU_METAL_REQUIRE_ICB_MDI). Adapter gating is restated in the CapabilitiesQuery idiom: feature-set tables plus family checks, with the paravirtual exclusion kept and the splice-motivated OS-version gates dropped. The count-buffer offset validation moves to gfx-rs#9679 with the rest of the count feature. Running the full suite on device also surfaced a latent bug in the previous revision: setSupportIndirectCommandBuffers(true) makes pipeline creation fail for shaders Metal rejects in ICBs (e.g. wgpu's own TextureBlitter fragment shader on Apple GPUs). Pipeline creation now retries without the flag and multi-draws recorded under such pipelines fall back to the per-draw loop. Generation encoders, ICBs, and argument buffers are labeled and the per-pass generation runs as one labeled compute encoder, so GPU captures in Xcode show one legible generation node per pass. All 471 wgpu-gpu tests pass on Apple M4 Max (modulo failures also present on trunk on the same machine).
Fixes from an adversarial pre-push review of the prepass rework: - Gate setSupportIndirectCommandBuffers on MTLMeshRenderPipelineDescriptor behind macOS 14 / iOS 17 / tvOS 18.1 / visionOS 2.1: the property is one OS generation newer there than mesh pipelines themselves, so macOS 13 / iOS 16 would hit an unrecognized selector. Mesh pipelines created without the flag fall back to the per-draw loop. - Raise the ICB availability floor to macOS 10.15 / iOS 13 / tvOS 13: inheritPipelineState needs iOS/tvOS 13, and useResource:usage: only participates in hazard tracking from macOS 10.15 / iOS 13, which the deferred generation relies on. - Fix the tvOS entry in the ICB feature-set table (tvOS_GPUFamily1_v2 admitted the Apple2-class A8; the floor is tvOS_GPUFamily2_v2 / A10X). - Drop the begin_render_pass debug_assert on the deferred queue: wgpu-core legally records discarded-surface fixup clears (render passes) in the internal pre-pass before draining the queue. - Cache generation-kernel compile failures so a driver that rejects the library doesn't recompile it on every multi-draw call. - Report the retry error (the pipeline's real problem) when creation fails both with and without the ICB flag, and log a breadcrumb when a pipeline is silently downgraded to non-ICB. - Use the stage-scoped useResource for the ICB-referenced index buffer. The ICB itself keeps the stage-less variant: it is consumed by command ingestion, not a shader stage, and the stage-scoped call was observed on M4 to make the executed draws render nothing. - Rename the stale MULTI_DRAW_INDIRECT_BIND_GROUP_FALLBACK test (bind groups no longer force a fallback; the test now verifies bind-group inheritance through the ICB path), document the test-count constant's coupling to ICB_MIN_DRAW_COUNT, document the hal contract in the changelog, and correct the ICB bind-count constant's rationale.
Rebuilt on the reworked gfx-rs#9640 prepass architecture: multi_draw_*_indirect_count (draw, indexed, and mesh-task) executes through a GPU-generated indirect command buffer driven by an execution range that a deferred kernel clamps to min(count, max_count), so the draw count is never read by the CPU. The range clamp and ICB generation ride the same internal pre-pass command buffer as indirect validation. Because MULTI_DRAW_INDIRECT_COUNT is advertised per adapter but some pipelines cannot execute inside an ICB (on current Apple silicon drivers, any fragment shader that accesses a texture through direct bindings is rejected — verified empirically on M4; textures bound through argument buffers are accepted, which matches the constraint Dawn is migrating to argument buffers to lift), count draws under such pipelines fall back to a GPU-clamped copy of the argument buffer consumed by a fixed-length per-draw loop. The count still never round-trips through the CPU. A new sampled-texture count test covers the fallback; the existing count readback tests were also verified against the fallback path directly on M4 hardware. Also adds wgpu_types::INDIRECT_BUFFER_OFFSET_ALIGNMENT (review asked for a named alignment rather than a hardcoded 4), validates the count buffer offset against it, restores the deno chromium-experimental- multi-draw-indirect surface, and documents the Metal behavior on the feature flag.
Rebuilt on the reworked gfx-rs#9640 prepass architecture: multi_draw_*_indirect_count (draw, indexed, and mesh-task) executes through a GPU-generated indirect command buffer driven by an execution range that a deferred kernel clamps to min(count, max_count), so the draw count is never read by the CPU. The range clamp and ICB generation ride the same internal pre-pass command buffer as indirect validation. Because MULTI_DRAW_INDIRECT_COUNT is advertised per adapter but some pipelines cannot execute inside an ICB (on current Apple silicon drivers, any fragment shader that accesses a texture through direct bindings is rejected — verified empirically on M4; textures bound through argument buffers are accepted, which matches the constraint Dawn is migrating to argument buffers to lift), count draws under such pipelines fall back to a GPU-clamped copy of the argument buffer consumed by a fixed-length per-draw loop. The count still never round-trips through the CPU. A new sampled-texture count test covers the fallback; the existing count readback tests were also verified against the fallback path directly on M4 hardware. Also adds wgpu_types::INDIRECT_BUFFER_OFFSET_ALIGNMENT (review asked for a named alignment rather than a hardcoded 4), validates the count buffer offset against it, restores the deno chromium-experimental- multi-draw-indirect surface, and documents the Metal behavior on the feature flag.
The prepass rework dropped watchOS from the ICB gating along with the splice-era OS-version gates, but watchOS support was genuine hardware capability, not a splice workaround. Restore it. Validated end-to-end on Apple Watch SE 2 (S8 SiP, watchOS 11.6): the GPU reports "Apple S4 GPU", supports the Apple5 family, and an indirect command buffer GPU-generated by the multi-draw generation kernel renders correctly via executeCommandsInBuffer: (readback + visual). objc2-metal exposes no MTLFeatureSet::watchOS_* values, so the feature-set table can't cover watchOS, and watchOS classifies as OsType::Ios here with no watchos key in family_check; the watch is therefore detected purely by GPU family, folded into icb_family_support behind available!(watchos = 11.0) (false on every other platform). watchOS 11 is the floor for the same reason iOS/iPadOS/tvOS 17.x are excluded: earlier drivers mishandle the render-ICB path.
68eb50d to
4ada51d
Compare
Fold the Apple Watch Apple5 detection into icb_family_support behind available!(watchos = 11.0) (which is false off watchOS), instead of a separate cfg(target_os = "watchos") branch. objc2-metal exposes no MTLFeatureSet::watchOS_* values so the feature-set table can't cover watchOS; the watch is detected purely by GPU family. Also raise the floor from watchOS 6 to 11, matching the iOS/tvOS 18-era minimum (17.x drivers mishandle the render-ICB path). Folds into gfx-rs#9640's watchOS commit on squash.
inner-daemons
left a comment
There was a problem hiding this comment.
Sorry for the delay I've been "busy" (procrastinating).
Code quality is very high here. I have a few comments/questions but nothing major. Hopefully we can land this soon.
| .unwrap(); | ||
| } | ||
|
|
||
| async fn mesh_multi_draw_indirect_color_readback(ctx: TestingContext) { |
There was a problem hiding this comment.
Always love more mesh shader tests, thx
| /// older families, but those have not been validated with wgpu's generation | ||
| /// kernels; widen this table as device validation lands. |
There was a problem hiding this comment.
In naga's xtask crate's validation logic, we have a command to automatically validate the MSL output with speciifc older OS versions, something like xcrun -sdk macosx metal -mmacosx-version-min=10.11 .... Maybe you can run this in CI on the metal files within wgpu-hal. So that we could support this on as many devices as possible when it lands. You're not obligated and it could come in a follow-up but I would really prefer to support all of the devices possible.
| /// discrete/Apple-silicon Mac). Apple documents render-ICB support on some | ||
| /// older families, but those have not been validated with wgpu's generation | ||
| /// kernels; widen this table as device validation lands. | ||
| const INDIRECT_COMMAND_BUFFERS_RENDERING_SUPPORT: &[MTLFeatureSet] = &[ |
There was a problem hiding this comment.
If you could leave links to documentation (perhaps in the wayback machine) where you got this that would be ideal.
| && if os_type == super::OsType::Macos { | ||
| device.supportsFamily(MTLGPUFamily::Mac2) | ||
| || device.supportsFamily(MTLGPUFamily::Metal3) | ||
| } else { | ||
| device.supportsFamily(MTLGPUFamily::Apple5) | ||
| || device.supportsFamily(MTLGPUFamily::Metal3) | ||
| || (os_type == super::OsType::Tvos | ||
| && device.supportsFamily(MTLGPUFamily::Apple3)) | ||
| }) |
There was a problem hiding this comment.
Why can't you just use the else block with an || device.supportsFamily(MTLGPUFamily::Mac2)?
| let indirect_command_buffers_rendering = !is_virtual | ||
| && icb_api_check | ||
| && (Self::supports_any(device, INDIRECT_COMMAND_BUFFERS_RENDERING_SUPPORT) | ||
| || icb_family_support); | ||
| let indirect_command_buffers_compute = !is_virtual | ||
| && icb_api_check | ||
| && (Self::supports_any(device, INDIRECT_COMMAND_BUFFERS_COMPUTE_SUPPORT) | ||
| || icb_family_support); |
There was a problem hiding this comment.
Might need another check and private capabilities for mesh shader indirect drawing. Looks like mesh shaders are supported on Apple7 and up but indirect drawing with them is apple9 and up.
| && match descriptor { | ||
| MetalGenericRenderPipelineDescriptor::Standard(_) => true, | ||
| MetalGenericRenderPipelineDescriptor::Mesh(_) => { | ||
| available!(macos = 14.0, ios = 17.0, tvos = 18.1, visionos = 2.1) |
There was a problem hiding this comment.
I highlighted this in a prior comment but I think this should be a private cap based on family checks, rather than an OS version check every time you need to know.
| let raw = match create(&descriptor) { | ||
| Ok(raw) => raw, | ||
| Err(first_err) if request_icb_support => { | ||
| descriptor.setSupportIndirectCommandBuffers(false); | ||
| supports_indirect_command_buffers = false; | ||
| let raw = create(&descriptor).map_err(|retry_err| { |
There was a problem hiding this comment.
I love apple and metal
| /// Minimum `draw_count` for which lowering a fixed-count multi-draw to an | ||
| /// indirect command buffer pays off. The ICB path costs an ICB allocation | ||
| /// plus reset/generate/optimize GPU work in the pre-pass, so below this | ||
| /// threshold the plain per-draw indirect loop is faster. Chosen empirically | ||
| /// on A12 through M4 hardware. |
There was a problem hiding this comment.
Glad to hear that this was gotten through empirical testing, appreciate the good work.
|
|
||
| impl IcbArgumentEncoderState { | ||
| fn new(pipeline: &IcbCommandPipeline) -> Self { | ||
| let encoder = unsafe { pipeline.function.newArgumentEncoderWithBufferIndex(0) }; |
There was a problem hiding this comment.
Looks to be a deprecated function: https://developer.apple.com/documentation/metal/mtlfunction/makeargumentencoder(bufferindex:reflection:)
|
|
||
| fn supports_icb_multi_draw(&self) -> bool { | ||
| self.shared.private_caps.indirect_command_buffers_rendering | ||
| && self.shared.private_caps.indirect_command_buffers_compute |
Connections
Motivated by BabylonNative WebGPU rendering where CPU-side multi-draw looping became visible in large batched AR scenes (AR-portal camera+render, Hill-Valley-style GLTF with baked/static PBR geometry and a WGSL SSAO pass). AR portal demo: https://doc.babylonjs.com/features/featuresDeepDive/webXR/webXRDemos#ar-demo
What this does
Lowers fixed-count
multi_draw_indirect/multi_draw_indexed_indirect(and the mesh-task variant) on Metal to GPU-generatedMTLIndirectCommandBuffers instead of looping over draws on the CPU. Applies when the draw count is ≥ 8; smaller batches keep the per-draw loop.Design
Generation runs in wgpu-core's internal "Pre Pass" command buffer — the same one that already carries indirect-draw validation — via a new default-no-op hal method,
CommandEncoder::encode_deferred_multi_draws:useResource+executeCommandsInBufferinto the open render encoder, then queues a generation request.inject_validation_pass, so generation reads validated arguments). Metal emits one reset blit, one labeled compute encoder holding every generation dispatch for the pass, and one optimize blit. The queue runs this buffer before the pass buffer, so the render encoder is never split.Because there is no render-encoder split, this needs none of the state capture/restore, store-action rewriting, pass-resumability gating, or environment variables an earlier revision of this PR used (net −700 lines vs. that revision). Adapter gating is feature-set tables + GPU-family checks + an API-availability floor (macOS 10.15 / iOS 13 / tvOS 13, for hazard-tracked
useResourceandinheritPipelineState), with Apple's paravirtual Metal device excluded (it advertises the feature sets but SIGABRTs executing render ICBs in CI) and acfg(target_os = "watchos")-gated Apple5 path (see Validation).Handled quirks:
supportIndirectCommandBuffersfor pipelines whose fragment shader samples a texture through direct bindings (argument-buffer-bound textures are accepted — the same constraint behind Dawn's argument-buffer migration). Pipeline creation retries without the flag and multi-draws recorded under such a pipeline take the per-draw loop.setSupportIndirectCommandBufferson the mesh-pipeline descriptor needs a newer OS than mesh pipelines themselves, so it is availability-gated (macOS 14 / iOS 17 / tvOS 18.1 / visionOS 2.1).dispatchThreadgroupssized bythreadExecutionWidth()with an in-shadercmdIndex >= drawCountguard (works on minimal GPUs that lack non-uniform dispatch); an explicit primitive-type mapping avoids relying onMTLPrimitiveTypematching MSLprimitive_typevalues.MULTI_DRAW_INDIRECT_COUNTis out of scope here and lands in #9679.Profiling
Generation encoders, ICBs, and argument buffers are labeled ("wgpu multi-draw ICB generation", etc.), and each render pass contributes exactly one generation compute encoder, so Xcode 26 GPU captures show one legible generation node per pass next to the unbroken render pass.
Validation
Local (M4 Max / Metal):
cargo fmt --check,cargo clippy -p wgpu-hal --features metal, and thewgpu-gpudraw_indirect/multi_draw/mesh_shadersuites pass (45 tests). The ICB path was instrumented to confirm it actually executes rather than silently falling back.On real hardware, a purpose-built headless harness renders a nanite/meshlet-style scene and validates by CPU readback of the presented surface — a 16-seed jump-flood over two 1024×1024
rgba32floatstorage textures across 10 GPU-only compute passes, feeding a GPU-generated meshletmulti_draw_indirect(1600 draws), at ≥ 32 MiB peak GPU memory. Path counters (patched into wgpu-hal for the run) prove the lowering:First-generation Apple TV 4K (A10X) is the meaningful run — render-ICB execution has historically been fragile there — and with generation hoisted out of the render pass it runs the full ICB path clean at 4K. The fragment-texture rejection above reproduces identically on M4 / A12 / A10X / A15 / S4.
The validation workload source is posted as a comment on #9679.
Squash or Rebase? Squash before merge; the branch keeps hardening iterations visible for review.
Checklist
wgpumay be affected behaviorally (if they enable MDI).CHANGELOG.mdentries are present.