Skip to content

[vk] implement our own descriptor allocator - #9161

Merged
cwfitzgerald merged 2 commits into
gfx-rs:trunkfrom
teoxoy:vk-desc-alloc
Apr 29, 2026
Merged

[vk] implement our own descriptor allocator#9161
cwfitzgerald merged 2 commits into
gfx-rs:trunkfrom
teoxoy:vk-desc-alloc

Conversation

@teoxoy

@teoxoy teoxoy commented Mar 4, 2026

Copy link
Copy Markdown
Member

Connections
Closes #9078.

Description
Replaces gpu-descriptor with our own descriptor allocator.

The pool growth strategy is the same as gpu-descriptor's.
The set allocation is different, gpu-descriptor tries to allocate from newer/larger/emptier pools. As opposed to the allocator in this PR which prefers smaller/older/fuller pools to prevent fragmentation.
Because the set allocation is different, the pool shrinking strategy is different, gpu-descriptor tries to destroy older pools, the allocator in this PR will destroy newer pools.

A more substantial difference is that the allocator in this PR will not immediately destroy empty pools (since they might have been recently created). It will wait until there is 1/4th its capacity in other pools.

Pools are also created and destroyed when BGLs are created/destroyed rather than deferring this to BG creation/destruction.

Testing
Existing tests.

Squash or Rebase?
n/a

@inner-daemons inner-daemons left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really high quality stuff! A few comments, you know the deal

Comment thread wgpu-hal/src/vulkan/descriptor.rs
}

#[derive(Default)]
struct Bucket {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe comment on exactly what many of these types are?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some more comments but if you have something more specific I should add let me know.

Comment thread wgpu-hal/src/vulkan/descriptor.rs
Comment thread wgpu-hal/src/vulkan/descriptor.rs Outdated
Comment thread wgpu-hal/src/vulkan/descriptor.rs Outdated
Comment on lines +215 to +220
// vkFreeDescriptorSets is documented to return:
// - VK_ERROR_UNKNOWN
// - VK_ERROR_VALIDATION_FAILED (we shouldn't encounter this one)
// wgpu-hal doesn't currently report errors in destroy functions.
// Panic here for now. It might be ok to ignore the error but the
// proper solution would probably be to lose the device.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this deserve a // TODO to lose the device instead of panicking in hal?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will open a new issue.

Comment thread wgpu-hal/src/vulkan/descriptor.rs
Comment thread wgpu-hal/src/vulkan/descriptor.rs Outdated
@teoxoy
teoxoy force-pushed the vk-desc-alloc branch 2 times, most recently from 69c4567 to b1fac16 Compare March 30, 2026 17:42
@teoxoy
teoxoy requested a review from inner-daemons March 30, 2026 17:44
@inner-daemons

Copy link
Copy Markdown
Collaborator

On my list to re-review.

@inner-daemons inner-daemons left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 opinions carrying over sorta from the last review, your choice on them. I didn't look over everything again, just gonna hand this off to someone with write permission.

Comment on lines +17 to +19
pub uniform_buffer_dynamic: u32,
pub storage_buffer: u32,
pub storage_buffer_dynamic: u32,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do still tihnk that the dynamic ones should have a comment explaining their purpose for being separate

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did add a comment on DescriptorCounts about this. Do you want me to expand on it?

}

#[derive(Debug, PartialEq, Eq, Hash)]
struct BucketKey {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably also warrants a comment on what exactly it corresponds to

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the key of the buckets HashMap. Do you have a suggestion for a doc comment? I thought its name would suffice.

@cwfitzgerald cwfitzgerald left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking generally good, one potential issue, which seems to be valid.

During my standard LLM audit of this it found a potential panic in the failure path:


Bug: register_layout failure panics in error cleanup

Summary

If register_layout fails during create_bind_group_layout, the error path calls destroy_bind_group_layout, which calls unregister_layout, which does .unwrap() on a bucket that was never inserted. This panics.

Trace

In device.rs, the error path after register_layout fails:

let result = self.desc_allocator.lock()
    .register_layout(&self.shared.raw, &layout);
if let Err(err) = result {
    unsafe { self.destroy_bind_group_layout(layout) };  // ← calls unregister_layout
    return Err(err);
}

In descriptor.rs, register_layout on the Vacant path:

Vacant(vacant_entry) => {
    let mut bucket = Bucket::default();
    bucket.create_pool(device, vacant_entry.key(), POOL_MIN_SETS)?;  // ← fails here
    vacant_entry.insert(bucket)  // ← never reached
}

The ? propagates before vacant_entry.insert(bucket), so the bucket is never inserted into self.buckets. Then unregister_layout does:

let bucket = self.buckets.get_mut(&key).unwrap();  // ← panics: no bucket for this key

Trigger conditions

Both must hold:

  1. First BGL with this (DescriptorCounts, update_after_bind) combination — takes the Vacant arm
  2. vkCreateDescriptorPool returns an errorVK_ERROR_OUT_OF_HOST_MEMORY, VK_ERROR_OUT_OF_DEVICE_MEMORY, or VK_ERROR_FRAGMENTATION

The Occupied path cannot fail (no Vulkan calls before layouts.insert), so this only affects new bucket keys.

Suggested fix

Don't route through destroy_bind_group_layout on registration failure, since the layout was never registered:

if let Err(err) = result {
    // register_layout failed — layout was never registered, so don't
    // call destroy_bind_group_layout (which would try to unregister).
    unsafe { self.shared.raw.destroy_descriptor_set_layout(layout.raw, None) };
    self.counters.bind_group_layouts.sub(1);
    return Err(err);
}

Making unregister_layout tolerant of missing buckets would also work but would mask genuine logic errors on the non-error paths.

@teoxoy

teoxoy commented Apr 22, 2026

Copy link
Copy Markdown
Member Author

@cwfitzgerald I fixed the issue.

@teoxoy
teoxoy requested a review from cwfitzgerald April 22, 2026 16:27

@cwfitzgerald cwfitzgerald left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good!

@cwfitzgerald
cwfitzgerald enabled auto-merge (squash) April 29, 2026 22:19
@cwfitzgerald
cwfitzgerald merged commit a03e866 into gfx-rs:trunk Apr 29, 2026
58 checks passed
@teoxoy
teoxoy deleted the vk-desc-alloc branch April 30, 2026 06:32
slyedoc pushed a commit to slyedoc/wgpu that referenced this pull request May 28, 2026
Co-authored-by: Connor Fitzgerald <connorwadefitzgerald@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants