|
| 1 | +# Rust generator backend — design notes |
| 2 | + |
| 3 | +> **Status:** implemented (GL/GLES) and proven. The backend lives in |
| 4 | +> `src/generator/rust/mod.rs` (`gloam … rust`); the design below is largely |
| 5 | +> realized. It emits a self-contained `#![no_std]` crate, detects extensions |
| 6 | +> with XXH3 (reusing the resolver's `Extension.hash`), and drives a real |
| 7 | +> triangle on hardware — see |
| 8 | +> [examples/rust/gl-triangle](../examples/rust/gl-triangle/). Size is at |
| 9 | +> parity with the C loader (below). mx-global is implemented as `--mx-global` |
| 10 | +> (free-function dispatch over a write-once `UnsafeCell` global — see the |
| 11 | +> note below on why not `OnceLock`). Vulkan is still out of scope. |
| 12 | +
|
| 13 | +## Goal & scope |
| 14 | + |
| 15 | +Add a **Rust generator backend** alongside the C backend, emitting a loader |
| 16 | +that is **more idiomatic than GLAD2's Rust output** without giving up gloam's |
| 17 | +small-code / fast-load characteristics. |
| 18 | + |
| 19 | +- **GL / GLES first**, targeting a Rust port of the `gl-triangle` example. |
| 20 | +- **Vulkan later** — its type system is the hard part (see |
| 21 | + [type translation](#type-translation-the-main-new-work)); GL is much easier. |
| 22 | + |
| 23 | +## What to avoid (the GLAD2 reference) |
| 24 | + |
| 25 | +A GLAD2 Rust loader (`glad --api gl:core,gles2 --merge rust --alias`) was |
| 26 | +examined at `D:\dev\vk-api-loader-shootout\tmp` as the anti-pattern. Its |
| 27 | +deficiencies (all of which gloam structurally avoids): |
| 28 | + |
| 29 | +- **Emits the entire registry** (every vendor extension) → ~15k lines, ~5,400 |
| 30 | + flat `pub const` enums. Its monolithic `load()` touches every function, so the |
| 31 | + whole registry is **pinned against dead-code elimination** once you load. |
| 32 | +- **Every function is a `static mut FnPtr { ptr, is_loaded: bool }`** — 16 bytes |
| 33 | + each (double a raw pointer), thousands of them, as global mutable state |
| 34 | + (unsound under edition 2024). |
| 35 | +- **transmute-per-call** through the global storage; unloaded slots point at a |
| 36 | + `not_initialized` panic stub. |
| 37 | +- **No selection, no packed name blob, no bulk/range loading** (one |
| 38 | + `GetProcAddress` per function by name), **no extension hashing** (string |
| 39 | + compares). Release `.rlib` ≈ 7.7 MiB (an rlib overstates linked size, but the |
| 40 | + DCE-pinning above means the effective bloat is real). |
| 41 | + |
| 42 | +## API shape (decisions) |
| 43 | + |
| 44 | +### Naming — keep GL short names, own the context |
| 45 | + |
| 46 | +- The generated type is **`Gl`** (UpperCamelCase — no lint fights); the caller's |
| 47 | + binding is conventionally **`gl`**; methods keep the **verbatim GL short name**. |
| 48 | + Call site reads `gl.DrawArrays(GL_TRIANGLES, 0, 3)` — namespace-prefix feel, |
| 49 | + original names, no rename transform. |
| 50 | +- A lowercase `struct gl` was rejected: it trips `non_camel_case_types`. The |
| 51 | + `Gl`-type + `gl`-binding pair gives the same reading with zero lint friction. |
| 52 | +- `.` (method on an owned value) is the default, not `::` (which implies a |
| 53 | + global / free function — see [dispatch modes](#dispatch-modes-mirror-c-mx--mx-global)). |
| 54 | + |
| 55 | +### Constants & enums — newtype with free typed consts |
| 56 | + |
| 57 | +- `#[repr(transparent)] pub struct GLenum(pub u32)` and a companion |
| 58 | + `GLbitfield` newtype with `BitOr`/`BitAnd` (for `GL_*_BIT` combining). |
| 59 | + `repr(transparent)` ⇒ ABI-identical to `u32`, zero runtime cost. |
| 60 | +- Constants are emitted as **free consts typed as the newtype**, glob-importable: |
| 61 | + ```rust |
| 62 | + pub const GL_TRIANGLES: GLenum = GLenum(0x0004); |
| 63 | + // use gloam_gl::consts::*; → gl.DrawArrays(GL_TRIANGLES, 0, 3) (bare name) |
| 64 | + ``` |
| 65 | + This gives bare `GL_*` names (no `GLenum::` prefix) **and** call-site type |
| 66 | + safety (a raw integer won't coerce to `GLenum`). Associated consts |
| 67 | + (`GLenum::TRIANGLES`) were the alternative; identical safety, but the free-const |
| 68 | + form matches the "namespace prefix, not a rename" preference and the C output. |
| 69 | +- **True per-group enums are punted indefinitely** — GL's `group=` metadata is |
| 70 | + too incomplete to be worth the untangling, and driver-returned values would |
| 71 | + risk invalid-discriminant UB. |
| 72 | +- **Constant-typing caveat (the real cost of newtypes):** each const must be |
| 73 | + assigned a newtype, but some GL values are polymorphic — `GL_ZERO`, `GL_ONE`, |
| 74 | + `GL_NONE` are used as both enum and integer. Decision: **default those to |
| 75 | + `GLenum`** and revisit if real API usage forces casts. This is a coarse |
| 76 | + enum/bitfield/int bucketing, far smaller than full per-group typing. |
| 77 | + |
| 78 | +### Safety — thin raw crate now, optional safe crate later |
| 79 | + |
| 80 | +- gloam generates the **low-level crate**: raw context, PFN table, `unsafe` |
| 81 | + dispatch methods, loader, enums, extension detection. Mechanical from |
| 82 | + `FeatureSet`, in scope. Ideally `#![no_std]` (core + the loader callback). |
| 83 | +- A **safe wrapper is a separate, hand-written, downstream crate** (the `-sys` + |
| 84 | + safe convention), at the user's discretion. Safe abstractions (object |
| 85 | + ownership, slice-vs-ptr+len, `glGetError`→`Result`, RAII) need human judgment |
| 86 | + that can't be reliably derived from XML. Generating mechanical safety is a |
| 87 | + possible *future* project, deliberately out of the initial scope. |
| 88 | + |
| 89 | +## Context representation (perf-critical) |
| 90 | + |
| 91 | +The rules here are what make Rust dispatch match C's `global + offset → PFN`: |
| 92 | + |
| 93 | +1. **PFN table stored inline** as a fixed `[Pfn; K]` (`K` known at generation |
| 94 | + time). **Never `Box<[…]>` / `Vec<…>`.** Inline ⇒ `self.pfns[IDX]` is one load |
| 95 | + `[base + const_offset]`. A boxed/vec table adds a second load (pointer in the |
| 96 | + struct → then the PFN) — the double-indirection to avoid. |
| 97 | +2. **`Gl` is `!Clone` and `!Copy`.** The inline table is tens of KB; deriving |
| 98 | + `Clone` would make `gl.clone()` a silent ~20 KB `memcpy`. Not deriving it |
| 99 | + forces sharing by `&Gl` / `Arc` / `static` — no accidental big copies. |
| 100 | +3. **`Arc<Gl>` adds no dispatch load.** `Arc → &data` is constant-offset pointer |
| 101 | + arithmetic, not a memory load; the inline PFN is then one load. Arc's only |
| 102 | + cost (atomic refcount) is on clone/drop, never on the hot path. (Arc is |
| 103 | + likely unnecessary, but it wouldn't harm dispatch.) |
| 104 | + |
| 105 | +Pointers are 8 bytes each (like C's `pfnArray`), not GLAD's 16-byte |
| 106 | +`FnPtr{ptr,bool}`. |
| 107 | + |
| 108 | +## Dispatch modes (mirror C `--mx` / `--mx-global`) |
| 109 | + |
| 110 | +- **`--mx` (explicit context):** owned `Gl`, `gl.DrawArrays(...)` methods, share |
| 111 | + `&Gl`. Single indirection via the receiver register. |
| 112 | +- **`--mx-global` (primary in gloam's philosophy), as implemented:** |
| 113 | + ```rust |
| 114 | + static GLOBAL: GlobalCell = ...; // UnsafeCell<Gl>, statically zero-filled |
| 115 | + unsafe { gl::load_gl_global(loader) }?; // or init_global(gl) with an owned Gl |
| 116 | + gl::DrawArrays(GL_TRIANGLES, 0, 3); // free fn over the global — the `::` form |
| 117 | + ``` |
| 118 | + Dispatch is `[fixed_base + offset] → PFN` — a single indirection from a |
| 119 | + link-time-fixed base, exactly like C, with **no branch and no atomic** on |
| 120 | + the hot path. |
| 121 | + - **Why not `OnceLock`** (the original sketch): the crate went `#![no_std]` |
| 122 | + (`OnceLock` is std-only), and its `get()` adds an is-initialized check C |
| 123 | + doesn't have. Instead the global is a **statically zero-filled `Gl`** in an |
| 124 | + `UnsafeCell`: reading flags before init is defined (everything reads |
| 125 | + absent), free presence queries stay *safe*, and only `init_global` is |
| 126 | + unsafe — its contract ("complete before, and never concurrent with, any |
| 127 | + other global access; publish via an ordinary happens-before edge") is the |
| 128 | + same discipline C's global already requires. Dispatching before init trips |
| 129 | + the dispatch `debug_assert` in debug builds. |
| 130 | + |
| 131 | +## Matching C's size / load-time |
| 132 | + |
| 133 | +Every mechanism behind gloam-C's small/fast loader has a zero-cost Rust form: |
| 134 | + |
| 135 | +| Mechanism | Rust form | |
| 136 | +|---|---| |
| 137 | +| API selection (only requested APIs) | already done upstream in `resolve/` | |
| 138 | +| 8-byte PFN slots, bulk **range loading** | inline `[Pfn; K]`, same `(start,count)` range tables | |
| 139 | +| Packed **function-name blob** + offsets | `static FN_NAMES: &[u8]` + `[u16; K]` — same `.rodata` | |
| 140 | +| **Extension hashing** (XXH3-64, sort, bsearch) | `static EXT_HASHES: [u64; M]` + small (vendored/no_std) xxh3 + `binary_search` — beats GLAD's string compares | |
| 141 | +| Dispatch wrappers | `#[inline]` methods melt into call sites | |
| 142 | +| Loader | take `&mut dyn FnMut(&CStr) -> *const c_void` (one instantiation, avoids monomorphization bloat) | |
| 143 | +| Release footprint | same knobs: `panic="abort"`, LTO, strip | |
| 144 | + |
| 145 | +`#[repr(C)] union` field access, if we ever want a typed named view over the |
| 146 | +same memory, is a **pure reinterpret — no runtime type/bounds check** (the only |
| 147 | +cost is the compile-time `unsafe`). So the C union trick is available at zero |
| 148 | +cost, though the inline array + const-indexed `get_unchecked` methods likely |
| 149 | +suffice. |
| 150 | + |
| 151 | +## Architecture: where it plugs in |
| 152 | + |
| 153 | +The resolve pipeline is backend-neutral; a Rust backend reuses all of it. See |
| 154 | +the survey in the module map ([CONTRIBUTING.md](../CONTRIBUTING.md)). |
| 155 | + |
| 156 | +**Reused as-is** (no changes): selection, requirements, merge batching, command |
| 157 | +ordering optimization, PFN range tables, alias pairs, topo sort, extension |
| 158 | +hashes, indices, **structured command params** (`Param { name, type_raw }`), |
| 159 | +bootstrap-command policy. |
| 160 | + |
| 161 | +**The seam** (small, additive): |
| 162 | +- `Generator::Rust(RustArgs)` variant in `src/cli.rs` (the subcommand *is* the |
| 163 | + backend selector — no `--language` flag). |
| 164 | +- `generator::rust::generate(fs, args, out, store, cmdline) -> GeneratedTree` |
| 165 | + mirroring `generator::c::generate`. |
| 166 | +- A match arm in `src/main.rs` (next to the `Generator::C` arm). |
| 167 | +- One `Generator::Rust(r) => r.alias` arm in `src/resolve/mod.rs` (the only |
| 168 | + existing coupling that reads the generator's alias flag). |
| 169 | + |
| 170 | +### Type translation (the main new work) |
| 171 | + |
| 172 | +The `FeatureSet` carries **C type text** in exactly three fields, which a Rust |
| 173 | +backend cannot emit directly and must translate: |
| 174 | + |
| 175 | +- `Param.type_raw` (e.g. `"const GLuint *"`) |
| 176 | +- `Command.return_type` (e.g. `"const GLubyte *"`) |
| 177 | +- `TypeDef.raw_c` (whole assembled C typedefs) |
| 178 | + |
| 179 | +There is **no existing type-mapping infrastructure**. So a Rust backend needs a |
| 180 | +**C-type → Rust-type translator**: |
| 181 | + |
| 182 | +- **GL/GLES: small.** ~40 base types, nearly all primitive aliases — |
| 183 | + `GLuint→u32`, `GLint→i32`, `GLenum→GLenum` (newtype), `GLfloat→f32`, |
| 184 | + `GLchar→c_char`, `GLboolean→u8`, `GLsizei→i32`, `GLintptr`/`GLsizeiptr→isize`, |
| 185 | + pointers→`*const`/`*mut`, `GLDEBUGPROC`→fn pointer. Calling convention is |
| 186 | + `extern "system"` (matches `APIENTRY`). Array params (`float v[4]`) are |
| 187 | + smuggled inside `type_raw` with the name embedded — must be special-cased. |
| 188 | +- **Vulkan: large.** `TypeDef.raw_c` holds full struct/union/handle/fn-pointer |
| 189 | + bodies; re-deriving those as Rust needs either a C-text parser or **richer IR |
| 190 | + exposure** (the structured struct members are flattened into `raw_c` before |
| 191 | + `FeatureSet`). This is the dominant reason Vulkan is deferred. |
| 192 | + |
| 193 | +The Rust backend also re-implements the C-policy bits from |
| 194 | +`generator/c/model.rs` (PFN-type names, parameter-string formatting) in Rust |
| 195 | +form, and its own emit layer (templates or direct codegen). |
| 196 | + |
| 197 | +## Results (what was proven) |
| 198 | + |
| 199 | +The original skepticism was size/load-time parity with C. Outcome, measured on |
| 200 | +the merged `gl:core=3.3,gles2=3.0` loader (2940 commands, 992 extensions), |
| 201 | +compiled to an object: |
| 202 | + |
| 203 | +| section | C `gl.o` | Rust loader | note | |
| 204 | +|---|---|---|---| |
| 205 | +| `.text` | 5,304 | ~850 | dispatch is `#[inline]`, so it lives at call sites (like C's macros), not in the loader object; Rust also skips the runtime Shellsort (its known table is pre-sorted) | |
| 206 | +| `.data`/`.rodata` | 93,570 | ~92,200 | packed name blob + XXH3 table; near-identical once both carry the ext-hash table | |
| 207 | +| `.bss` | 24,552 | 0 | owned-context mode; an `OnceLock` global would add ~24 KB, matching C | |
| 208 | + |
| 209 | +So: **at parity, no structural bloat.** Every early regression (fat-pointer |
| 210 | +name table; tuple-padded hash table) traced to a representation choice that was |
| 211 | +fixed back to ≤ C. The full 2940-method surface added *nothing* to the compiled |
| 212 | +object (inline methods materialize only when called). |
| 213 | + |
| 214 | +Other items, all settled: |
| 215 | + |
| 216 | +- **The C→Rust type translator holds across the entire GL/GLES surface** |
| 217 | + (`cargo check` clean over all 2940 signatures, incl. opaque handles, CL |
| 218 | + structs, all five callback typedefs). |
| 219 | +- **Extension detection matches C** — XXH3 reused from `Extension.hash`, hashed |
| 220 | + at runtime with `xxhash_rust::xxh3::xxh3_64`; a mock-driver test and the live |
| 221 | + hardware run both confirm it. |
| 222 | +- **Constant typing** — polymorphic `GL_ZERO`/`GL_ONE`/`GL_NONE` default to |
| 223 | + `GLenum`; no friction observed in the example. |
| 224 | +- **Real rendering** — [examples/rust/gl-triangle](../examples/rust/gl-triangle/) |
| 225 | + draws a triangle via winit + glutin on a desktop GL 3.3 context (verified on |
| 226 | + an NVIDIA RTX 5090), plus a headless `--ci` pixel check. |
| 227 | + |
| 228 | +## Deliberate omissions |
| 229 | + |
| 230 | +- **No `--loader` (dlopen/LoadLibrary) layer.** In C that layer papers over a |
| 231 | + real portability gap; in Rust the established idiom is the downstream crate |
| 232 | + choosing [`libloading`](https://crates.io/crates/libloading) (or its |
| 233 | + windowing stack's `get_proc_address`, as the example does). Emitting one |
| 234 | + would also drag platform `dlopen` bindings into an otherwise `#![no_std]` |
| 235 | + crate. Intentionally omitted, not deferred. |
| 236 | + |
| 237 | +## Open decisions |
| 238 | + |
| 239 | +- **Emitter style: direct string emission vs templates.** The C backend |
| 240 | + renders minijinja templates; the Rust backend emits via `format!`/`push_str` |
| 241 | + with escaped literals. The string form is compiler-adjacent (no template |
| 242 | + runtime, rustc catches malformed interpolations) but the `\x20`-indented |
| 243 | + literals are harder to scan and review than `.j2` files. Fine at the |
| 244 | + current ~1100 lines; **revisit before Vulkan**, which would multiply the |
| 245 | + emit surface (structs, unions, handles, PFN typedefs). |
| 246 | + |
| 247 | +## Deferred |
| 248 | + |
| 249 | +- **Vulkan** — the `TypeDef.raw_c` struct/handle bodies are the hard part. |
0 commit comments