|
| 1 | +# Contributing to gloam |
| 2 | + |
| 3 | +## Architecture overview |
| 4 | + |
| 5 | +The pipeline is strictly linear: **parse -> IR -> resolve -> generate**. |
| 6 | + |
| 7 | +``` |
| 8 | +CLI args |
| 9 | + -> fetch/bundled XML specs |
| 10 | + -> parse into RawSpec (IR) |
| 11 | + -> resolve into FeatureSet (indexed, sorted, grouped) |
| 12 | + -> generate C code via minijinja templates |
| 13 | +``` |
| 14 | + |
| 15 | +### Module map |
| 16 | + |
| 17 | +| Module | Purpose | |
| 18 | +|---|---| |
| 19 | +| `main.rs` | Entry point; orchestrates CLI -> resolve -> generate | |
| 20 | +| `cli.rs` | clap-derived CLI definitions, `ApiRequest` parsing, extension filter parsing | |
| 21 | +| `ir.rs` | Raw intermediate representation types directly from XML (pre-resolution) | |
| 22 | +| `bundled.rs` | Compile-time-embedded XML specs and auxiliary headers (`include_str!`) | |
| 23 | +| `fetch.rs` | Load specs from bundled copies or remote Khronos URLs (`--fetch`) | |
| 24 | +| `build_info.rs` | Git version metadata embedded at compile time (generated by `build.rs`) | |
| 25 | +| `preamble.rs` | Copyright/license/provenance comment block for generated files | |
| 26 | +| **`parse/`** | **XML -> `RawSpec` IR** | |
| 27 | +| `parse/mod.rs` | Orchestrator; raw C text extraction, enum value computation | |
| 28 | +| `parse/types.rs` | `<types>` -> `RawType[]` with topological sort (Kahn's algorithm) | |
| 29 | +| `parse/enums.rs` | `<enums>` -> flat enums + Vulkan typed enum groups | |
| 30 | +| `parse/commands.rs` | `<commands>` -> `RawCommand[]` with alias-chain fixup and Vulkan scope inference | |
| 31 | +| `parse/features.rs` | `<feature>` + `<extension>` -> `RawFeature[]` + `RawExtension[]` | |
| 32 | +| **`resolve/`** | **`RawSpec` + CLI args -> `FeatureSet`** | |
| 33 | +| `resolve/mod.rs` | Three-phase orchestrator (selection -> materialization -> grouping) | |
| 34 | +| `resolve/types.rs` | Public output types: `FeatureSet`, `Feature`, `Extension`, `Command`, etc. | |
| 35 | +| `resolve/selection.rs` | Which features/extensions are "in" (filter, promoted, predecessors) | |
| 36 | +| `resolve/requirements.rs` | Collects required types/enums/commands from selected features+extensions | |
| 37 | +| `resolve/commands.rs` | Indexed `Command` entries, PFN ordering optimization, alias pair extraction | |
| 38 | +| `resolve/pfn.rs` | PFN range tables (feature -> command index ranges) | |
| 39 | +| `resolve/enums.rs` | Flat enums and Vulkan enum groups for the resolved set | |
| 40 | +| `resolve/typedefs.rs` | Type list with dependency ordering and include-guard inference | |
| 41 | +| `resolve/protect.rs` | Platform protection lattice; group-by-protection coalescing | |
| 42 | +| `resolve/spec_info.rs` | Spec-level constants: display names, PFN prefixes, name prefixes | |
| 43 | +| **`generator/`** | **`FeatureSet` -> output files** | |
| 44 | +| `generator/c/mod.rs` | C generator: template rendering, function-name blob layout, aux header copying | |
| 45 | +| `generator/c/templates/` | Minijinja templates (`header.h.j2`, `source.c.j2`, `hash_search.j2`, `library.j2`, `loader.j2`, etc.) | |
| 46 | + |
| 47 | +### Key data types |
| 48 | + |
| 49 | +- **`RawSpec`** (`ir.rs`): Everything parsed from one XML spec family (types, |
| 50 | + enums, commands, features, extensions). This is the pre-resolution IR. |
| 51 | +- **`FeatureSet`** (`resolve/types.rs`): Fully indexed, sorted, |
| 52 | + protection-grouped output ready for template rendering. Contains features, |
| 53 | + extensions, commands, types, flat enums, enum groups, PFN range tables, |
| 54 | + extension index subsets, and protection-grouped lists. |
| 55 | +- **`Command`** (`resolve/types.rs`): Indexed command with short name, PFN type |
| 56 | + name, parameter string, scope (for Vulkan), and protection. |
| 57 | +- **`Protection`** (`resolve/protect.rs`): `Unconditional` or `Guarded(macros)` |
| 58 | + — used to emit `#ifdef` blocks in generated code. |
| 59 | + |
| 60 | +### Resolution phases |
| 61 | + |
| 62 | +1. **Selection**: Determine which API features and extensions are included |
| 63 | + based on CLI flags (`--api`, `--extensions`, `--promoted`, `--predecessors`, |
| 64 | + `--baseline`). |
| 65 | +2. **Materialization**: Build indexed arrays of commands, types, enums. |
| 66 | + Optimize command ordering to minimize PFN range fragmentation. Build PFN |
| 67 | + range tables. |
| 68 | +3. **Grouping**: Coalesce items by platform protection for `#ifdef`-correct |
| 69 | + header emission. |
| 70 | + |
| 71 | +## Extension detection strategy |
| 72 | + |
| 73 | +At load time, the generated code detects driver-supported extensions |
| 74 | +without any string comparisons: |
| 75 | + |
| 76 | +1. Calls `glGetIntegerv(GL_NUM_EXTENSIONS, &n)` to get the count. |
| 77 | +2. Calls `glGetStringi(GL_EXTENSIONS, i)` for each `i`, hashes each name |
| 78 | + with XXH3-64 (the same algorithm used at generator time), and stores |
| 79 | + the hashes in a heap-allocated `uint64_t[]`. |
| 80 | +3. Shellsorts the array in-place (Ciura gap sequence — no extra memory, |
| 81 | + ~160 bytes of code). |
| 82 | +4. Binary-searches the sorted driver hashes against the pre-baked known |
| 83 | + extension hash table embedded in the generated source. |
| 84 | + |
| 85 | +This gives O(n log n) total work to detect all extensions, with O(log n) |
| 86 | +per lookup. |
| 87 | + |
| 88 | +## Alias resolution |
| 89 | + |
| 90 | +When `--alias` is passed, the generated loader emits a runtime resolver: |
| 91 | +after loading all function pointers, if the canonical slot for an alias |
| 92 | +pair is null but the alias slot was loaded by the driver (or vice versa), |
| 93 | +the loaded pointer is propagated to both slots. This handles the case |
| 94 | +where a driver only exposes one spelling of a promoted function. |
| 95 | + |
| 96 | +`--alias` is a *runtime* concern — it does not affect which extensions |
| 97 | +are selected. For selection-time alias expansion see `--promoted` and |
| 98 | +`--predecessors`. |
| 99 | + |
| 100 | +## Generated code internals |
| 101 | + |
| 102 | +### Function name blob |
| 103 | + |
| 104 | +All function names are packed into a single `kFnNameData[]` string |
| 105 | +table with pre-computed offsets in `kFnNameOffsets[]`. This is indexed |
| 106 | +in lockstep with `pfnArray[]` so that loading code can look up the name |
| 107 | +for any function pointer slot by index. |
| 108 | + |
| 109 | +### PFN range tables |
| 110 | + |
| 111 | +Features and extensions map to contiguous ranges of command indices. |
| 112 | +The tables are contiguous-run compressed: each entry is a |
| 113 | +`(start_index, count)` pair. This allows bulk-loading all function |
| 114 | +pointers for a feature or extension with a single loop over the range. |
| 115 | + |
| 116 | +### Platform guards |
| 117 | + |
| 118 | +Extensions that require platform-specific headers are wrapped in |
| 119 | +`#ifdef` blocks (e.g. `VK_USE_PLATFORM_WIN32_KHR`). The resolver |
| 120 | +groups consecutive items with identical protection into |
| 121 | +`ProtectedGroup<T>` to minimize the number of `#ifdef`/`#endif` pairs |
| 122 | +in the generated output. |
| 123 | + |
| 124 | +## Khronos XML spec gotchas |
| 125 | + |
| 126 | +The XML specs have numerous inconsistencies that the parser handles |
| 127 | +explicitly. These are documented in code comments as "Spec gotcha #N": |
| 128 | + |
| 129 | +1. Command alias entries lack full prototypes — walk alias chains to copy |
| 130 | + signatures |
| 131 | +2. Type and enum dependency ordering requires topological sort |
| 132 | +3. Vulkan enum groups can be empty after filtering — prune them |
| 133 | +4. Bitwidth=64 must propagate through enum alias chains |
| 134 | +5. GL has auto-excluded types that should be silently skipped |
| 135 | +6. Some XML contains C++ `//` comments — rewrite to `/* */` for C99 |
| 136 | +7. macOS needs a special `ptrdiff_t` guard for GL pointer-sized types |
| 137 | +8. `GLX_SGIX_video_source` and `GLX_SGIX_dmbuffer` are broken — silently drop |
| 138 | +9. `WGL_ARB_extensions_string` is mandatory but might be missing — warn |
| 139 | +11. Vulkan `<enums type="enum"|"bitmask">` should not be re-processed as flat |
| 140 | + constants |
| 141 | +12. Vulkan command scope is inferred from first parameter type |
| 142 | + (Global/Instance/Device) |
| 143 | +13. Duplicate enum names with conflicting values must be detected and rejected |
| 144 | + |
| 145 | +## Testing |
| 146 | + |
| 147 | +Integration tests live in `tests/`. They invoke the binary via |
| 148 | +`assert_cmd`, generate into temp directories, and optionally compile the |
| 149 | +output with `cc` when available. |
| 150 | + |
| 151 | +| Test file | Coverage | |
| 152 | +|---|---| |
| 153 | +| `generate_c.rs` | GL/GLES C loader generation + compilation | |
| 154 | +| `generate_vulkan.rs` | Vulkan-specific generation | |
| 155 | +| `generate_wgl_glx.rs` | WGL, GLX, and cross-API edge cases | |
| 156 | +| `predecessor_promoted.rs` | `--promoted` and `--predecessors` flag behavior | |
| 157 | + |
| 158 | +Run the full suite: |
| 159 | + |
| 160 | +```sh |
| 161 | +cargo test |
| 162 | +``` |
| 163 | + |
| 164 | +## Style notes |
| 165 | + |
| 166 | +- **Determinism**: `IndexMap` is used throughout for insertion-order |
| 167 | + preservation. Never introduce `HashMap` iteration or other sources of |
| 168 | + non-determinism. |
| 169 | +- **Self-contained binary**: XML specs and auxiliary headers are embedded via |
| 170 | + `include_str!` from `bundled/`. |
| 171 | +- **Progress output**: goes to stderr, respects `--quiet`. Only errors use |
| 172 | + `eprintln!` unconditionally. |
| 173 | +- **Preamble**: generated files include the exact command line, gloam version, |
| 174 | + extension selection summary, and license notices. |
| 175 | +- **Minimal changes**: prefer small, focused changes. Don't over-abstract or |
| 176 | + add speculative infrastructure. |
0 commit comments