`_from_raw` and `_from_hal` APIs bypass validation, so wgpu should offer validated alternatives

[jimblandy]

We've been seeing a number of bugs caused by incorrect use of foo_from_raw APIs, which construct a higher-level resource from a lower-level resource. I'm referring to APIs like these:

• wgpu_hal::vulkan::Adapter::device_from_raw builds a wgpu_hal::vulkan::Device from an ash Vulkan device.
• wgpu_core::instance::Global::create_device_from_hal builds a wgpu_core::device::Device from wgpu_hal::DynOpenDevice.

There are similar functions for other sorts of resources like buffers and textures. The names vary, but I'm going to collectively call all of these _from_raw functions.

These _from_raw functions have legitimate uses, and sometimes there may not be any practical alternative. For example, if one wishes to use wgpu to manage resources created by some other part of the system, the _from_raw pattern seems like your only option.

However, in many cases, people are forced to use _from_raw functions for resources they've just created themselves, simply because they need to pass the creation operation some information specific to the particular backend they're using. But in doing so, they bypass the validation that would normally be carried out by wgpu-core. In this way, the present API design creates pressure on users to skip the validation that is one of wgpu's selling points. (Ideally, there would be a cross-platform wgpu-core feature for their case, but designing and implementing such a thing may not always be feasible.)

One solution is to let the users invoke the validation explicitly themselves, as is done in #9979.

Another solution we might consider is inspired by Vulkan's extending structures, but uses standard Rust features to do this in a type-safe way. Here, users would continue to call wgpu_core methods for resource creation as usual. However, these methods would accept a Vec<Box<dyn Any>> (empty in the normal case) which they would pass through unexamined to wgpu_hal. The wgpu_hal backend would then downcast the vector's elements to specific types it recognizes, and use them to inform resource creation. If an element had an unrecognized type, the backend would panic.

This effectively allows wgpu_core functions, which are backend-independent, to accept backend-specific types. Users would exercise exactly the same wgpu-core code paths as normal resource creation. I've verified that it's even possible to pass extending types by value, so that the user can transfer ownership of resources through wgpu-core to wgpu-hal. No unsafe code is required.

[jimblandy]

Here's an unsurprising example showing what this sort of code looks like. It demonstrates the recommended panic behavior, but you can tweak main to exercise both code paths.

use std::any::Any;

struct This {
    this_name: String,
}

struct That {
    that_name: String,
}

fn print_my_structs_by_value(objs: Vec<Box<dyn Any>>) {
    for obj in objs {
        let obj = match obj.downcast::<This>() {
            Ok(this) => {
                println!("This: {}", this.this_name);
                continue;
            }
            Err(other) => other
        };

        let obj = match obj.downcast::<That>() {
            Ok(that) => {
                println!("That: {}", that.that_name);
                continue;
            }
            Err(other) => other
        };

        panic!("Unexpected type in vector: {:?}", obj.type_id());
    }
}

fn owned() {
    let v = vec![
        Box::new(This { this_name: "Harvey".into() }) as Box<dyn Any>,
        Box::new(That { that_name: "Gertrude".into() }) as Box<dyn Any>,
        Box::new("Eustace") as Box<dyn Any>,
    ];

    print_my_structs_by_value(v);
}

fn print_my_structs_by_ref(objs: &[Box<dyn Any>]) {
    for obj in objs {
        if let Some(this) = obj.downcast_ref::<This>() {
            println!("&This: {}", this.this_name);
        } else if let Some(that) = obj.downcast_ref::<That>() {
            println!("&That: {}", that.that_name);
        } else {
            panic!("Unexpected type in slice: {:?}", obj.type_id());
        }
    }
}

fn by_ref() {
    let v = vec![
        Box::new(This { this_name: "Harvey".into() }) as Box<dyn Any>,
        Box::new(That { that_name: "Gertrude".into() }) as Box<dyn Any>,
        Box::new("Eustace") as Box<dyn Any>,
    ];

    print_my_structs_by_ref(&v);
}

fn main() {
    by_ref();
    owned();
}

[jimblandy]

There is also the classic type-map crate, which might be a nicer user experience. However, it doesn't have an is_empty method, so it would be impossible for backends to tell if they've consumed everything that the user passed, which seems like an important footgun to eliminate.

[jimblandy]

It probably makes sense to allow extending structures on most resource creation calls.

For request_device and device-specific resource creation calls, the backend is known, so the rule can be that the caller must only pass extension structures appropriate to that backend.

For request_adapter, however, it's not known yet what backend will be selected. The caller will need to label each extension structure with which backend is expected to handle it, or perhaps pass a HashMap<Backend, Vec<Box<dyn Any>>> (like Vec, an empty HashMap doesn't incur any heap allocations).

I don't think the caller generally knows which backends are going to end up being enumerated, either, so we'd have to ignore extension structures for backends we didn't enumerate.

Callers might also want to reject adapters that can't handle a given extension structure; or prefer adapters that can handle a given structure. For these behaviors, the backend will need a way to communicate with wgpu-core's iteration over adapters.