Design native WebGPU 3D architecture

[Sportinger]

Migrated from issue #62, which was removed from the issue tracker to keep broad architecture direction in Discussions.

Topic

General 3D rendering on the GPU without making Three.js the core dependency.

Context

MasterSelects has a long-term goal of supporting many media/file types as visual signals, including 3D models, Gaussian splats, point clouds, CAD-like formats, and other geometry/data sources.

The original issue title captured an architecture preference: build general 3D support directly on the app's GPU/WebGPU architecture instead of centering the editor around Three.js.

Possible Goals

• Treat 3D assets as first-class timeline/media layers.
• Keep 3D rendering integrated with the existing WebGPU compositor, effects, transitions, export, and project serialization.
• Avoid a hard architectural dependency on Three.js for core rendering paths.
• Allow format-specific loaders or helper libraries where useful, while preserving a native render path.
• Support future 3D formats such as glTF/GLB, OBJ, FBX, Gaussian splats, and point clouds.

Open Questions

• Which 3D format should be the first vertical slice?
• Should Three.js be avoided entirely, or only avoided as the core renderer?
• How should cameras, lights, materials, and transforms map onto timeline clips?
• How should 3D layers composite with 2D/video/text/vector layers?
• Which operations belong in WebGPU shaders vs helper libraries?
• How should export and preview parity be maintained?

Recommendation: use this discussion to shape the 3D architecture. Open focused implementation issues only for concrete slices such as a glTF layer, Gaussian splat renderer, or camera/transform UI.