Add mesh shading info to naga IR

[inner-daemons]

Connections
Mostly split off from #7930
Works towards #7197

Description
This PR adds mesh shading info to naga IR so that parsers and writers have an interface to use.

Testing
No testing yet (coming in later PRs, the code here has been tested in #7930)

Squash or Rebase?

Squash

Checklist

• Run cargo fmt.
• Run taplo format.
• Run cargo clippy --tests. If applicable, add:
  • --target wasm32-unknown-unknown
• Run cargo xtask test to run tests.
• If this contains user-facing changes, add a CHANGELOG.md entry.

[inner-daemons]

I'm unsure why the MSRV minimal versions thing is failing, it doesn't look related to this PR since it happens in another crate and this PR doesn't touch anything cargo.

[cwfitzgerald]

Should be fixed with #8112, thanks @andyleiserson!

[inner-daemons]

Going to ping you again @jimblandy just to make sure that this doesn't get forgotten about until the next meeting :)

[jimblandy]

@SupaMaggie70Incorporated I'll try to get to this tonight, but it may be Saturday.

[jimblandy]

@SupaMaggie70Incorporated Are these comments at the top of mesh_shading.md still accurate?

Currently naga has no support for mesh shaders beyond recognizing the additional shader stages. For this reason, all shaders must be created with Device::create_shader_module_passthrough.

[inner-daemons]

@SupaMaggie70Incorporated Are these comments at the top of mesh_shading.md still accurate?

Currently naga has no support for mesh shaders beyond recognizing the additional shader stages. For this reason, all shaders must be created with Device::create_shader_module_passthrough.

I've updated the top of mesh_shading.md. Until further PRs it remains true that you will need to use passthrough shaders but it now has more "processing" ability so the statement wasn't strictly true.

[jimblandy]

(This is not a review comment on this PR, and not directed at SupaMaggie, just commenting on the pre-existing state of the docs.)

This is not a thorough explanation of mesh shading and how it works. Those wishing to understand mesh shading more broadly should look elsewhere first.

This is just not adequate. If we're going to add syntax to Naga that is not covered by the WGSL specification, we need to document that syntax. Knowing how mesh shading works in Vulkan does not magically explain the syntax for mesh shaders in WGSL, or how to invoke them in wgpu.

[inner-daemons]

(This is not a review comment on this PR, and not directed at SupaMaggie, just commenting on the pre-existing state of the docs.)

This is not a thorough explanation of mesh shading and how it works. Those wishing to understand mesh shading more broadly should look elsewhere first.

This is just not adequate. If we're going to add syntax to Naga that is not covered by the WGSL specification, we need to document that syntax. Knowing how mesh shading works in Vulkan does not magically explain the syntax for mesh shaders in WGSL, or how to invoke them in wgpu.

It wouldn't be part of this PR, but if you think that having a writeup somewhere to describe mesh shaders would be useful, I'm happy to do that separately. However, I have never written a comprehensive GPU API spec before, so I don't exactly think it would be of comparable quality to e.g. the webgpu spec :)

Also, that specific disclaimer was copy-pasted directly from the RT spec.

[jimblandy]

It wouldn't be part of this PR, but if you think that having a writeup somewhere to describe mesh shaders would be useful, I'm happy to do that separately. However, I have never written a comprehensive GPU API spec before, so I don't exactly think it would be of comparable quality to e.g. the webgpu spec :)

Well, the alternative is just saying "read the code and the examples and figure it out". You want people to actually use your work, right?

I just pushed some docs; do they look correct?

[inner-daemons]

Just put up #8370, which is the WGSL parsing PR. The WGSL parsing and validation passes there, so this PR's validation seems to not be disallowing valid shaders yet.

[inner-daemons]

@jimblandy Me and @cwfitzgerald agreed that mesh shader outputs should always be buffered, so I'm gonna remove references in the spec to having to call setMeshOutputs exactly once before calling setVertex/setPrimitive and stuff like that.

[jimblandy]

@jimblandy Me and @cwfitzgerald agreed that mesh shader outputs should always be buffered, so I'm gonna remove references in the spec to having to call setMeshOutputs exactly once before calling setVertex/setPrimitive and stuff like that.

Does this mean we could also potentially have a design where the vertex and primitive arrays and counts are returned by the mesh shader? It seemed to me like much of this stuff would work just as well as new @builtin kinds:

struct MeshOutput {
    @builtin(vertex_count) num_vertices: u32,
    @builtin(primitive_count) num_primitives: u32,
    @builtin(vertices) vertices: array<Vertex, 10>,
    @builtin(primitives) primitives: array<Primitive, 15>,
}

@mesh @workgroup_size(15, 1, 1)
fn mesh_shader(
    @builtin(local_invocation_index) invocation: u32,
    @builtin(task_payload) payload: Payload
) -> MeshOutput {
    ...
}

[inner-daemons]

@jimblandy Me and @cwfitzgerald agreed that mesh shader outputs should always be buffered, so I'm gonna remove references in the spec to having to call setMeshOutputs exactly once before calling setVertex/setPrimitive and stuff like that.

Does this mean we could also potentially have a design where the vertex and primitive arrays and counts are returned by the mesh shader? It seemed to me like much of this stuff would work just as well as new @builtin kinds:

struct MeshOutput {

    @builtin(vertex_count) num_vertices: u32,

    @builtin(primitive_count) num_primitives: u32,

    @builtin(vertices) vertices: array<Vertex, 10>,

    @builtin(primitives) primitives: array<Primitive, 15>,

}



@mesh @workgroup_size(15, 1, 1)

fn mesh_shader(

    @builtin(local_invocation_index) invocation: u32,

    @builtin(task_payload) payload: Payload

) -> MeshOutput {

    ...

}

Err, that's actually really clever. I do think that this isn't right though, since the thing output is per workgroup and this would have each thread return a giant structure, that would in theory be the same for all threads anyway. I'm not opposed to something closer to this though, like a global thread group variable that holds the buffered outputs and then the mesh shader can declare that it uses that global variable.

[inner-daemons]

Ok, I've thought about it some more. I think that we can't really avoid buffering for another reason: the output format isn't guaranteed to match whatever the naga struct is. SPIR-V for example has arcane rules on which values must be in different output arrays. HLSL is very similar. I still don't think we should have the task payload/mesh outputs be per-thread, but a small tweak to your proposal would be as follows:

struct MeshOutput {
    @builtin(vertex_count) num_vertices: u32,
    @builtin(primitive_count) num_primitives: u32,
    @builtin(vertices) vertices: array<Vertex, 10>,
    @builtin(primitives) primitives: array<Primitive, 15>,
}

var<mesh_output> output: MeshOutput;
var<task_payload> task_payload: Payload;

@mesh(output) @workgroup_size(15, 1, 1)
fn mesh_shader(
    @builtin(local_invocation_index) invocation: u32,
) {
    ...
}

Then we would just need to allow these @builtin attributes on fields in a mesh_output variable. This variable would then probably function basically the same as task_payload does in task shaders, where only 1 can be used by a task shader and it must match what was declared, and other stages can't access such a global variable.

[jimblandy]

@inner-daemons Have the commits I've pushed been looking okay?

[inner-daemons]

@jimblandy They look great! I very much appreciate them.

[inner-daemons]

@jimblandy Do you have opinions about what we should go with? We could completely switch it up for the new option, which would probably mean less work on each of the backends. On the other hand, I would have to redo most of the work for mesh shaders, and there is a lot of work to redo.

I'm of the opinion that unless there's some problem with the new option I haven't seen we should go with that, but it would be kind of heartbreaking to restart after writing 7k lines and being blocked for 2 months on this PR.

[cwfitzgerald]

Been meaning to sit down with this and figure it out, but ended up getting sick over the weekend ;(

[jimblandy]

Writing down what we said in our meeting today:

• This PR is in no way blocked on the new suggestion for the API. Let's get this landed and iterate.
• It seems like the new suggestion might simplify SPIR-V generation.
• We should make sure our design doesn't hurt the performance of SPIR-V input.
• The WGSL front end work should wait until [wgsl-in] Implement template list discovery and handle all type resolution in the lowerer #8386 has landed.

[cwfitzgerald]

I'm tired of looking at this, you're tired of looking at this, I don't see any immediate problems, lets land and iterate!

Stale

[LukasKalbertodt]

Sorry for the spam, but I just want to say: thank you all for the work on this! I know this is not the first nor last step in the mesh shader journey, but as a usually quiet bystander I felt I should express my gratitude here :)