Skip to content

use Metal's Indirect Command Buffers for true GPU-side multi draw indirect - #9640

Open
matthargett wants to merge 30 commits into
gfx-rs:trunkfrom
rebeckerspecialties:metal-icb-multi-draw-indirect
Open

use Metal's Indirect Command Buffers for true GPU-side multi draw indirect#9640
matthargett wants to merge 30 commits into
gfx-rs:trunkfrom
rebeckerspecialties:metal-icb-multi-draw-indirect

Conversation

@matthargett

@matthargett matthargett commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Connections

Motivated by BabylonNative WebGPU rendering where CPU-side multi-draw looping became visible in large batched AR scenes (AR-portal camera+render, Hill-Valley-style GLTF with baked/static PBR geometry and a WGSL SSAO pass). AR portal demo: https://doc.babylonjs.com/features/featuresDeepDive/webXR/webXRDemos#ar-demo

What this does

Lowers fixed-count multi_draw_indirect / multi_draw_indexed_indirect (and the mesh-task variant) on Metal to GPU-generated MTLIndirectCommandBuffers instead of looping over draws on the CPU. Applies when the draw count is ≥ 8; smaller batches keep the per-draw loop.

Design

Generation runs in wgpu-core's internal "Pre Pass" command buffer — the same one that already carries indirect-draw validation — via a new default-no-op hal method, CommandEncoder::encode_deferred_multi_draws:

  • At the draw site, Metal allocates the ICB and records useResource + executeCommandsInBuffer into the open render encoder, then queues a generation request.
  • At pass end, wgpu-core calls the hal hook while recording the pre-pass buffer (right after inject_validation_pass, so generation reads validated arguments). Metal emits one reset blit, one labeled compute encoder holding every generation dispatch for the pass, and one optimize blit. The queue runs this buffer before the pass buffer, so the render encoder is never split.

Because there is no render-encoder split, this needs none of the state capture/restore, store-action rewriting, pass-resumability gating, or environment variables an earlier revision of this PR used (net −700 lines vs. that revision). Adapter gating is feature-set tables + GPU-family checks + an API-availability floor (macOS 10.15 / iOS 13 / tvOS 13, for hazard-tracked useResource and inheritPipelineState), with Apple's paravirtual Metal device excluded (it advertises the feature sets but SIGABRTs executing render ICBs in CI) and a cfg(target_os = "watchos")-gated Apple5 path (see Validation).

Handled quirks:

  • Current Apple-silicon drivers reject supportIndirectCommandBuffers for pipelines whose fragment shader samples a texture through direct bindings (argument-buffer-bound textures are accepted — the same constraint behind Dawn's argument-buffer migration). Pipeline creation retries without the flag and multi-draws recorded under such a pipeline take the per-draw loop.
  • setSupportIndirectCommandBuffers on the mesh-pipeline descriptor needs a newer OS than mesh pipelines themselves, so it is availability-gated (macOS 14 / iOS 17 / tvOS 18.1 / visionOS 2.1).
  • ICB generation uses uniform dispatchThreadgroups sized by threadExecutionWidth() with an in-shader cmdIndex >= drawCount guard (works on minimal GPUs that lack non-uniform dispatch); an explicit primitive-type mapping avoids relying on MTLPrimitiveType matching MSL primitive_type values.

MULTI_DRAW_INDIRECT_COUNT is out of scope here and lands in #9679.

Profiling

Generation encoders, ICBs, and argument buffers are labeled ("wgpu multi-draw ICB generation", etc.), and each render pass contributes exactly one generation compute encoder, so Xcode 26 GPU captures show one legible generation node per pass next to the unbroken render pass.

Validation

Local (M4 Max / Metal): cargo fmt --check, cargo clippy -p wgpu-hal --features metal, and the wgpu-gpu draw_indirect / multi_draw / mesh_shader suites pass (45 tests). The ICB path was instrumented to confirm it actually executes rather than silently falling back.

On real hardware, a purpose-built headless harness renders a nanite/meshlet-style scene and validates by CPU readback of the presented surface — a 16-seed jump-flood over two 1024×1024 rgba32float storage textures across 10 GPU-only compute passes, feeding a GPU-generated meshlet multi_draw_indirect (1600 draws), at ≥ 32 MiB peak GPU memory. Path counters (patched into wgpu-hal for the run) prove the lowering:

Device GPU OS Resolution multi-draw → ICB per-draw-loop fallback pixels
MacBook Pro M4 Max (Metal3) macOS 26 1200×1600 0
iPhone XS Max A12 (Apple5) iOS 18.7.9 1125×2436 0
Apple TV 4K gen 1 A10X (Apple3) tvOS 26.5 3840×2160 0
Apple TV 4K gen 3 A15 (Apple8) tvOS 26.6 3840×2160 0
Apple Watch SE 2 Apple S4 (Apple5) watchOS 11.6 160×160 0

First-generation Apple TV 4K (A10X) is the meaningful run — render-ICB execution has historically been fragile there — and with generation hoisted out of the render pass it runs the full ICB path clean at 4K. The fragment-texture rejection above reproduces identically on M4 / A12 / A10X / A15 / S4.

The validation workload source is posted as a comment on #9679.

Squash or Rebase? Squash before merge; the branch keeps hardening iterations visible for review.

Checklist

  • I self-reviewed and fully understand this PR.
  • WebGPU implementations built with wgpu may be affected behaviorally (if they enable MDI).
  • Validation and feature gates confine behavioral changes.
  • Tests demonstrate the altered logic works.
  • CHANGELOG.md entries are present.
  • The PR is minimal.
  • Commits are logically scoped and individually reviewable.
  • The description has enough context to understand the motivation and solution.

@inner-daemons

Copy link
Copy Markdown
Collaborator

Holy shit this is large. I will take a look but it might be a while.

In the meantime,

  • Does this address multi_draw_indirect_count and friends?
  • Is this implemented and tested for mesh shaders?
  • Does it handle the draw_index builtin?
  • Is there extensive testing otherwise?

Also, I must ask, to what extent is the code written by LLMs? What about the PR description?

@inner-daemons inner-daemons self-assigned this Jun 5, 2026
@inner-daemons
inner-daemons self-requested a review June 5, 2026 06:13
@matthargett

Copy link
Copy Markdown
Contributor Author

Holy shit this is large. I will take a look but it might be a while.

Sorry, I did try to keep it to the thinnest meaningful slice that demonstrated e2e uplift in my AR portal on real devices. if you see a natural seam I can split things out on, or I should disabuse myself of the constraint I imposed on myelf that a single PR bringing measurable uplift, just let me know! :D

* Does this address multi_draw_indirect_count and friends?

I didn't deal with multi_draw_indirect_count / multi_draw_indexed_indirect_count yet, as one of the ways to keep the PR's size down. It is intentionally scoped to "just" fixed-count multi_draw_indirect and multi_draw_indexed_indirect. The count-buffer variants have additional semantics and could/should be a separate follow-up.

* Is this implemented and tested for mesh shaders?

Mesh shaders are supported on my local M4 MacBook device, and I verified that existing fixed-count mesh MDI tests pass, but this PR doesn't add a Metal ICB mesh-command generation path. I did try to control scope, so count-buffer MDI and draw_index would need to be follow-ons unless it needs to be one atomic PR/commit from your perspective. (side note: I found mesh shaders to be pretty finicky on 26.0 operating systems when using MSL directly, so I'd anticipate even more physical device testing and iteration)

* Does it handle the draw_index builtin?

nope. In my notebook when I reviewed wgpu, I wrote that Naga’s MSL backend still rejects that builtin. (if that's changed or I was mistaken, do tell me!) would it be useful for me to add a test/fallback assertion so the ICB path cannot give the wrong impression on what it supports?

* Is there extensive testing otherwise?

the hardest part of this stuff is all the on-device testing (especially with ICBs which can halt/panic a device), and I try to capture all the quirks I ran into so others can avoid the tediousness of some of it on legacy devices that don't receive active update.
for testing, I:

  1. made a small MdiIcbTest program that I ran on all the Apple devices I have that verified correct values upon CPU readback
  2. I also integration tested it in the AR portal playground app that runs on my fork of BabylonNative that uses wgpu-native (instead of the GLES-based bgfx, which they don't want to switch from)
  3. I've been building a WASM interpreter-based fantasy console that uses wgpu-native, and I used both indirect draws and indirect compute generated on the GPU to drive some very neat 3D graphics demos but also pushing MOD/S3M and SNES SPC sample/song decoding onto the GPU. on the iPhone XS in particular, I had to push as much of the S3M music player's tracker and sample/effects math onto the GPU and use MDI to have decent quality and low battery draw by avoiding the A12 CPU cores.
    5.. I tested that on iPhone XS/12, iPad Pro (gen 2), M4 MacBook (not AR), AppleTV 4K 3rd gen (not AR), and Apple Vision Pro, and iterated until I got the performance I was hoping for from MDI. Let me know if you want links to supporting repos, or if reviewers would like to do a video call (or in-person meetup in San Francisco).

Also, I must ask, to what extent is the code written by LLMs? What about the PR description?

I came up with the high-level design for the WASM fantasy console in my notebooks, and MDI was added into that requirements list when I first learned about it at the Khronos meetup at GDC a few years ago. Then I used codex and iterated with it over the course of a few months to push the boundaries of older/weaker Apple GPUs (namely, iPhone XS and Apple Watch SE2/6).

I built up quite a local patch stack (which I reviewed with my own eyes at each step along the way) while creating all these demonstrations (WebAudio WASI using WebGPU in WASM worklets), and then directed my attention back to BabylonJS/BabylonNative which my startup's main product was written in. I had tried several times last year to fix bgfx's swapchains so we could ship a native BabylonJS app to the AVP app store, but then realized I just needed to rip off the bandaid and do the work to integrate wgpu-native. I used Codex to drive that work, but going through all their screenshot tests manually and spot checking accuracy was not left to AI.

Similarly, with the AR portal integration test, I had to physically pick up iPhones and iPads to scan the room after each build (so ARkit could detect the floor), walk into an out of the portal, note problems, and iterate with Codex. Then, I directed Codex to measure e2e framerate/CPU stats, alongside XCode CPU profiler data (L1/prefetch/branch-prediction misses), and guided it to the optimization result of 60 fps camera feed+3D AR projection on iPhone 12. (Some of those optimizations were on the Babylon.js side, and several of those submitted PRs have been merged/released by the nice folks at Microsoft.)

wrt the PR text: I directed Codex to summarize the important hard-won knowledge not represented in the diff, and gave it the bullet outline that it elaborated, but I also edited the PR title and description in the staging PR in my wgpu fork. I then had Claude Code review the fork PR description, commits, etc to look for problems but also ways it could better conform to gfx-rs project PR/issue norms/templates. If you look at my open source contributions across the last ~30 years, I can be very verbose especially when it comes to performance, exploits, and optimization.

So yes, I used multiple AI coding tools, but I personally reviewed and guided each step of the way, and did not post a PR in this project until it looked good (and passed CI) in my fork. Sorry for the wall of text, but I wanted to give a nuanced response so you (and whomever else) can understand this wasn't a one-shot AI prompt on a whim: I have things built with wgpu that I'd like to ship, and getting better FPS per milliwatt of power draw means MDI is key.

@inner-daemons

Copy link
Copy Markdown
Collaborator

Ok thank you for the explanations!

@inner-daemons

Copy link
Copy Markdown
Collaborator

Looking at this on my phone so pardon any misunderstandings:

  • Mesh shader draw calls are very similar to normal and indexed draw calls, I'm of the opinion that this PR should probably add the same features for all 3 draw calls "families"
  • Do you know how difficult it would be to add support for the draw index built in on top of this? If you don't know that's fine, we can worry about it later
  • Similar for multi draw indirect count & friends
  • AI usage is fine, I just like to know what I'm working with, especially for long descriptions, since AI has a tendency to write summaries of its actions to retroactively justify mistakes and try to portray them as sensibly as possible

@inner-daemons

Copy link
Copy Markdown
Collaborator

Also, you should add testing to wgpu's own test suite for every new piece of api surface, if you haven't already.

I would also like to see benchmarks that these indirect command buffers do actually speed things up, though I don't doubt it. This is not a requirement though

Add Metal ICB generation for mesh multi-draw indirect and opt-in count-buffer variants.

Wire the Chromium experimental multi-draw indirect API surface for CTS, add wgpu-owned readback tests for normal/indexed count-buffer draws and mesh MDI, and validate draw-count-buffer offset alignment.
@matthargett

Copy link
Copy Markdown
Contributor Author

Looking at this on my phone so pardon any misunderstandings:

* Mesh shader draw calls are very similar to normal and indexed draw calls, I'm of the opinion that this PR should probably add the same features for all 3 draw calls "families"

okay, based on our discord discussion and running more of the Dawn CTS, I expanded the test coverage and added some fallback code for when the feature isn't supported. Now ppl won't be surprised on Metal, and the accurate exception did traverse to my crash reporter (Sentry) in my integration native app

* Do you know how difficult it would be to add support for the draw index built in on top of this? If you don't know that's fine, we can worry about it later

It should be sort of straightforward, modulo discovering silicon/driver quirks across the test devices.

* Similar for multi draw indirect count & friends

same as above, should be straightforward. I can submit these PRs in parallel (based on eachother's PR branch), if you want to see more of the e2e all at once without the first PR being giant.

* AI usage is fine, I just like to know what I'm working with, especially for long descriptions, since AI has a tendency to write summaries of its actions to retroactively justify mistakes and try to portray them as sensibly as possible

yea, I see that in other repos as well. anyone I've worked with will tell you my biggest weakness is being verbose in my comms. I review everything in my fork before I ever submit work to upstream repos.

@inner-daemons

Copy link
Copy Markdown
Collaborator

For anyone else curious, I talked privately with Matt Hargett and he seems like a real and very experienced person.

same as above, should be straightforward. I can submit these PRs in parallel (based on eachother's PR branch), if you want to see more of the e2e all at once without the first PR being giant.

Not your obligation. I was just not sure if this approach would have to be redone for either of those to be implemented in the future, but you don't need to implement those right now.

@inner-daemons inner-daemons left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The final comment here is my main concern. I am happy to have this but that is a serious problem that needs to be addressed.

Also, I think that you were a little too willing to use environment variables. We usually don't accept environment variables as the only way to control behavior, especially in hal. And these cases are more hacky from what I understand.

Overall, I look forward to having this PR in eventually, and I'm grateful to you for putting in the effort to move towards that.

Comment thread tests/tests/wgpu-gpu/mesh_shader/mod.rs
Comment thread tests/tests/wgpu-gpu/draw_indirect.rs
Comment thread deno_webgpu/adapter.rs Outdated
Comment thread wgpu-core/src/command/render.rs Outdated
Comment on lines +927 to +928
#[error("Indirect draw count buffer offset {0:?} is not a multiple of 4")]
UnalignedIndirectCountBufferOffset(BufferAddress),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO there should be a limit for the indirect args alignment, rather than a hardcoded value.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — added wgpu_types::INDIRECT_BUFFER_OFFSET_ALIGNMENT and both the indirect-offset and the new count-offset checks use it (in #9679, where the count validation now lives). Since the value is fixed at 4 by the WebGPU spec and no backend requires stricter, I went with a named constant à la COPY_BUFFER_ALIGNMENT rather than a queryable limit — happy to promote it to a real Limits entry if you'd prefer it queryable per adapter.

Comment thread wgpu-hal/src/metal/adapter.rs Outdated
Comment thread wgpu-hal/src/metal/adapter.rs Outdated
Comment thread wgpu-hal/src/metal/adapter.rs Outdated
Comment thread wgpu-hal/src/metal/command.rs Outdated
Comment thread wgpu-hal/src/metal/command.rs Outdated
Comment thread wgpu-hal/src/metal/command.rs
…aw-indirect

# Conflicts:
#	CHANGELOG.md
#	deno_webgpu/render_pass.rs
Extract the inline MSL generation kernels into wgpu-hal/src/metal/shaders/
and include_str! them, following the gles/shaders precedent, and document
the ICB tuning and primitive-topology constants.

Requested in review of gfx-rs#9640.
Document why MULTI_DRAW_INDIRECT_COUNT is exposed past all_webgpu_mask():
it surfaces as Chromium's experimental
chromium-experimental-multi-draw-indirect WebGPU feature.

Requested in review of gfx-rs#9640.
Move the MULTI_DRAW_INDIRECT_COUNT surface (deno bindings and the
chromium-experimental-multi-draw-indirect mapping) out to gfx-rs#9679, which
owns the count feature.

Requested in review: the env-gated feature exposure didn't belong in
this PR.
Rework requested in review of gfx-rs#9640: instead of suspending and resuming
the render encoder around each multi-draw, the ICB is allocated and its
execution recorded at draw time, and the reset/generate/optimize work is
deferred into the same internal pre-pass command buffer that carries
indirect-draw validation, via a new default-no-op hal method
CommandEncoder::encode_deferred_multi_draws. The queue executes that
buffer before the pass, so the pass is never interrupted and no render
state needs to be captured or restored.

This deletes the suspend/resume machinery, the render-state shadow
tracking, the store-action rewriting, the pass-resumability gating, and
both environment variables (WGPU_METAL_FORCE_ICB_MDI,
WGPU_METAL_REQUIRE_ICB_MDI). Adapter gating is restated in the
CapabilitiesQuery idiom: feature-set tables plus family checks, with the
paravirtual exclusion kept and the splice-motivated OS-version gates
dropped. The count-buffer offset validation moves to gfx-rs#9679 with the rest
of the count feature.

Running the full suite on device also surfaced a latent bug in the
previous revision: setSupportIndirectCommandBuffers(true) makes pipeline
creation fail for shaders Metal rejects in ICBs (e.g. wgpu's own
TextureBlitter fragment shader on Apple GPUs). Pipeline creation now
retries without the flag and multi-draws recorded under such pipelines
fall back to the per-draw loop.

Generation encoders, ICBs, and argument buffers are labeled and the
per-pass generation runs as one labeled compute encoder, so GPU captures
in Xcode show one legible generation node per pass.

All 471 wgpu-gpu tests pass on Apple M4 Max (modulo failures also
present on trunk on the same machine).
Fixes from an adversarial pre-push review of the prepass rework:

- Gate setSupportIndirectCommandBuffers on MTLMeshRenderPipelineDescriptor
  behind macOS 14 / iOS 17 / tvOS 18.1 / visionOS 2.1: the property is one
  OS generation newer there than mesh pipelines themselves, so macOS 13 /
  iOS 16 would hit an unrecognized selector. Mesh pipelines created without
  the flag fall back to the per-draw loop.
- Raise the ICB availability floor to macOS 10.15 / iOS 13 / tvOS 13:
  inheritPipelineState needs iOS/tvOS 13, and useResource:usage: only
  participates in hazard tracking from macOS 10.15 / iOS 13, which the
  deferred generation relies on.
- Fix the tvOS entry in the ICB feature-set table (tvOS_GPUFamily1_v2
  admitted the Apple2-class A8; the floor is tvOS_GPUFamily2_v2 / A10X).
- Drop the begin_render_pass debug_assert on the deferred queue: wgpu-core
  legally records discarded-surface fixup clears (render passes) in the
  internal pre-pass before draining the queue.
- Cache generation-kernel compile failures so a driver that rejects the
  library doesn't recompile it on every multi-draw call.
- Report the retry error (the pipeline's real problem) when creation fails
  both with and without the ICB flag, and log a breadcrumb when a pipeline
  is silently downgraded to non-ICB.
- Use the stage-scoped useResource for the ICB-referenced index buffer.
  The ICB itself keeps the stage-less variant: it is consumed by command
  ingestion, not a shader stage, and the stage-scoped call was observed on
  M4 to make the executed draws render nothing.
- Rename the stale MULTI_DRAW_INDIRECT_BIND_GROUP_FALLBACK test (bind
  groups no longer force a fallback; the test now verifies bind-group
  inheritance through the ICB path), document the test-count constant's
  coupling to ICB_MIN_DRAW_COUNT, document the hal contract in the
  changelog, and correct the ICB bind-count constant's rationale.
matthargett added a commit to rebeckerspecialties/wgpu that referenced this pull request Jul 8, 2026
Rebuilt on the reworked gfx-rs#9640 prepass architecture:

multi_draw_*_indirect_count (draw, indexed, and mesh-task) executes
through a GPU-generated indirect command buffer driven by an execution
range that a deferred kernel clamps to min(count, max_count), so the
draw count is never read by the CPU. The range clamp and ICB generation
ride the same internal pre-pass command buffer as indirect validation.

Because MULTI_DRAW_INDIRECT_COUNT is advertised per adapter but some
pipelines cannot execute inside an ICB (on current Apple silicon
drivers, any fragment shader that accesses a texture through direct
bindings is rejected — verified empirically on M4; textures bound
through argument buffers are accepted, which matches the constraint
Dawn is migrating to argument buffers to lift), count draws under such
pipelines fall back to a GPU-clamped copy of the argument buffer
consumed by a fixed-length per-draw loop. The count still never
round-trips through the CPU. A new sampled-texture count test covers
the fallback; the existing count readback tests were also verified
against the fallback path directly on M4 hardware.

Also adds wgpu_types::INDIRECT_BUFFER_OFFSET_ALIGNMENT (review asked
for a named alignment rather than a hardcoded 4), validates the count
buffer offset against it, restores the deno chromium-experimental-
multi-draw-indirect surface, and documents the Metal behavior on the
feature flag.
matthargett added a commit to rebeckerspecialties/wgpu that referenced this pull request Jul 8, 2026
Rebuilt on the reworked gfx-rs#9640 prepass architecture:

multi_draw_*_indirect_count (draw, indexed, and mesh-task) executes
through a GPU-generated indirect command buffer driven by an execution
range that a deferred kernel clamps to min(count, max_count), so the
draw count is never read by the CPU. The range clamp and ICB generation
ride the same internal pre-pass command buffer as indirect validation.

Because MULTI_DRAW_INDIRECT_COUNT is advertised per adapter but some
pipelines cannot execute inside an ICB (on current Apple silicon
drivers, any fragment shader that accesses a texture through direct
bindings is rejected — verified empirically on M4; textures bound
through argument buffers are accepted, which matches the constraint
Dawn is migrating to argument buffers to lift), count draws under such
pipelines fall back to a GPU-clamped copy of the argument buffer
consumed by a fixed-length per-draw loop. The count still never
round-trips through the CPU. A new sampled-texture count test covers
the fallback; the existing count readback tests were also verified
against the fallback path directly on M4 hardware.

Also adds wgpu_types::INDIRECT_BUFFER_OFFSET_ALIGNMENT (review asked
for a named alignment rather than a hardcoded 4), validates the count
buffer offset against it, restores the deno chromium-experimental-
multi-draw-indirect surface, and documents the Metal behavior on the
feature flag.
The prepass rework dropped watchOS from the ICB gating along with the
splice-era OS-version gates, but watchOS support was genuine hardware
capability, not a splice workaround. Restore it.

Validated end-to-end on Apple Watch SE 2 (S8 SiP, watchOS 11.6): the GPU
reports "Apple S4 GPU", supports the Apple5 family, and an indirect
command buffer GPU-generated by the multi-draw generation kernel renders
correctly via executeCommandsInBuffer: (readback + visual).

objc2-metal exposes no MTLFeatureSet::watchOS_* values, so the feature-set
table can't cover watchOS, and watchOS classifies as OsType::Ios here with
no watchos key in family_check; the watch is therefore detected purely by
GPU family, folded into icb_family_support behind available!(watchos =
11.0) (false on every other platform). watchOS 11 is the floor for the
same reason iOS/iPadOS/tvOS 17.x are excluded: earlier drivers mishandle
the render-ICB path.
@matthargett
matthargett force-pushed the metal-icb-multi-draw-indirect branch from 68eb50d to 4ada51d Compare July 9, 2026 00:12
matthargett added a commit to rebeckerspecialties/wgpu that referenced this pull request Jul 9, 2026
Fold the Apple Watch Apple5 detection into icb_family_support behind
available!(watchos = 11.0) (which is false off watchOS), instead of a
separate cfg(target_os = "watchos") branch. objc2-metal exposes no
MTLFeatureSet::watchOS_* values so the feature-set table can't cover
watchOS; the watch is detected purely by GPU family. Also raise the
floor from watchOS 6 to 11, matching the iOS/tvOS 18-era minimum (17.x
drivers mishandle the render-ICB path).

Folds into gfx-rs#9640's watchOS commit on squash.

@inner-daemons inner-daemons left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay I've been "busy" (procrastinating).

Code quality is very high here. I have a few comments/questions but nothing major. Hopefully we can land this soon.

.unwrap();
}

async fn mesh_multi_draw_indirect_color_readback(ctx: TestingContext) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always love more mesh shader tests, thx

Comment on lines +586 to +587
/// older families, but those have not been validated with wgpu's generation
/// kernels; widen this table as device validation lands.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In naga's xtask crate's validation logic, we have a command to automatically validate the MSL output with speciifc older OS versions, something like xcrun -sdk macosx metal -mmacosx-version-min=10.11 .... Maybe you can run this in CI on the metal files within wgpu-hal. So that we could support this on as many devices as possible when it lands. You're not obligated and it could come in a follow-up but I would really prefer to support all of the devices possible.

/// discrete/Apple-silicon Mac). Apple documents render-ICB support on some
/// older families, but those have not been validated with wgpu's generation
/// kernels; widen this table as device validation lands.
const INDIRECT_COMMAND_BUFFERS_RENDERING_SUPPORT: &[MTLFeatureSet] = &[

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you could leave links to documentation (perhaps in the wayback machine) where you got this that would be ideal.

Comment on lines +744 to +752
&& if os_type == super::OsType::Macos {
device.supportsFamily(MTLGPUFamily::Mac2)
|| device.supportsFamily(MTLGPUFamily::Metal3)
} else {
device.supportsFamily(MTLGPUFamily::Apple5)
|| device.supportsFamily(MTLGPUFamily::Metal3)
|| (os_type == super::OsType::Tvos
&& device.supportsFamily(MTLGPUFamily::Apple3))
})

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't you just use the else block with an || device.supportsFamily(MTLGPUFamily::Mac2)?

Comment on lines +761 to +768
let indirect_command_buffers_rendering = !is_virtual
&& icb_api_check
&& (Self::supports_any(device, INDIRECT_COMMAND_BUFFERS_RENDERING_SUPPORT)
|| icb_family_support);
let indirect_command_buffers_compute = !is_virtual
&& icb_api_check
&& (Self::supports_any(device, INDIRECT_COMMAND_BUFFERS_COMPUTE_SUPPORT)
|| icb_family_support);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might need another check and private capabilities for mesh shader indirect drawing. Looks like mesh shaders are supported on Apple7 and up but indirect drawing with them is apple9 and up.

&& match descriptor {
MetalGenericRenderPipelineDescriptor::Standard(_) => true,
MetalGenericRenderPipelineDescriptor::Mesh(_) => {
available!(macos = 14.0, ios = 17.0, tvos = 18.1, visionos = 2.1)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I highlighted this in a prior comment but I think this should be a private cap based on family checks, rather than an OS version check every time you need to know.

Comment on lines +1788 to +1793
let raw = match create(&descriptor) {
Ok(raw) => raw,
Err(first_err) if request_icb_support => {
descriptor.setSupportIndirectCommandBuffers(false);
supports_indirect_command_buffers = false;
let raw = create(&descriptor).map_err(|retry_err| {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love apple and metal

Comment on lines +36 to +40
/// Minimum `draw_count` for which lowering a fixed-count multi-draw to an
/// indirect command buffer pays off. The ICB path costs an ICB allocation
/// plus reset/generate/optimize GPU work in the pre-pass, so below this
/// threshold the plain per-draw indirect loop is faster. Chosen empirically
/// on A12 through M4 hardware.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to hear that this was gotten through empirical testing, appreciate the good work.


impl IcbArgumentEncoderState {
fn new(pipeline: &IcbCommandPipeline) -> Self {
let encoder = unsafe { pipeline.function.newArgumentEncoderWithBufferIndex(0) };

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


fn supports_icb_multi_draw(&self) -> bool {
self.shared.private_caps.indirect_command_buffers_rendering
&& self.shared.private_caps.indirect_command_buffers_compute

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend: metal Issues with Metal

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants