WASM user-defined functions and extensions#6256
Open
glommer wants to merge 2 commits intotursodatabase:mainfrom
Open
WASM user-defined functions and extensions#6256glommer wants to merge 2 commits intotursodatabase:mainfrom
glommer wants to merge 2 commits intotursodatabase:mainfrom
Conversation
Preparation for the next commit which adds a non-Copy field (Arc<dyn WasmRuntimeApi>) to DatabaseOpts. This is a mechanical change that adds .clone() calls at all sites where DatabaseOpts was previously implicitly copied. Split into its own commit to keep the WASM feature commit focused on the actual feature. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce a system for extending Turso with custom scalar functions, types, and virtual tables written in WebAssembly. Users register WASM modules via two new SQL statements: CREATE FUNCTION name LANGUAGE wasm AS X'...' EXPORT 'export_name'; CREATE EXTENSION name LANGUAGE wasm AS X'...'; Single functions (CREATE FUNCTION) export one callable with the signature (argc: i32, argv: i32) -> i64, where arguments are marshalled into the module's linear memory via a bump allocator (turso_malloc). Extensions (CREATE EXTENSION) export a turso_ext_init entry point that returns a JSON manifest declaring multiple functions, custom types, and virtual tables, all registered in a single statement. Unknown manifest fields are rejected so that modules targeting a newer Turso version fail explicitly rather than silently losing resources. The core design decision is a pluggable runtime: the database engine defines a WasmRuntimeApi trait but bundles no WASM executor. Each host environment injects its own: - CLI and Rust SDK: Wasmtime, with epoch-based interruption and fuel metering to bound runaway UDFs. Gated behind --unstable-wasm (CLI) and with_unstable_wasm_runtime() (SDK). - Node.js / Bun: the host engine's built-in WebAssembly API via N-API, adding zero dependency weight. - Python / Go / .NET / Java: Wasmtime compiled into the native binding. This means the core can be built with no WASM dependency at all, and JavaScript environments pay no overhead since they reuse V8 or JSC natively. WASM instances are cached per-connection in an LRU keyed by module name, with a shared atomic byte budget (default 10 MiB) across all connections on a database. The cache avoids recompilation on repeated calls while bounding total memory. A Rust SDK crate (turso-wasm-sdk) provides a #[turso_wasm] proc macro that transforms natural Rust functions into WASM exports with automatic ABI marshalling for i64, f64, &str, &[u8], Option<T>, and return types. A C shim (sqlite3_wasm_shim) bridges the standard sqlite3 extension API (sqlite3_create_function, sqlite3_value_*, sqlite3_result_*) so that unmodified C extensions can be compiled to WASM with wasi-sdk and loaded via CREATE EXTENSION. Function and extension definitions are persisted in the schema (turso_master) and re-registered on database open. DROP FUNCTION and DROP EXTENSION clean up the schema, symbol table, and runtime modules. The entire feature is marked unstable: CLI requires --unstable-wasm, all SDK entry points use unstable-prefixed names, and documentation carries experimental warnings. Only scalar functions are supported; aggregate and window WASM functions are not yet available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add WASM user-defined functions and extensions