Skip to content

WASM user-defined functions and extensions#6256

Open
glommer wants to merge 2 commits intotursodatabase:mainfrom
glommer:turso-wasm-udf
Open

WASM user-defined functions and extensions#6256
glommer wants to merge 2 commits intotursodatabase:mainfrom
glommer:turso-wasm-udf

Conversation

@glommer
Copy link
Copy Markdown
Contributor

@glommer glommer commented Apr 5, 2026

Add WASM user-defined functions and extensions

Introduce a system for extending Turso with custom scalar functions,
types, and virtual tables written in WebAssembly. Users register WASM
modules via two new SQL statements:

  CREATE FUNCTION name LANGUAGE wasm AS X'...' EXPORT 'export_name';
  CREATE EXTENSION name LANGUAGE wasm AS X'...';

Single functions (CREATE FUNCTION) export one callable with the
signature (argc: i32, argv: i32) -> i64, where arguments are
marshalled into the module's linear memory via a bump allocator
(turso_malloc). Extensions (CREATE EXTENSION) export a turso_ext_init
entry point that returns a JSON manifest declaring multiple functions,
custom types, and virtual tables, all registered in a single statement.
Unknown manifest fields are rejected so that modules targeting a newer
Turso version fail explicitly rather than silently losing resources.

The core design decision is a pluggable runtime: the database engine
defines a WasmRuntimeApi trait but bundles no WASM executor. Each host
environment injects its own:

 - CLI and Rust SDK: Wasmtime, with epoch-based interruption and fuel
   metering to bound runaway UDFs. Gated behind --unstable-wasm (CLI)
   and with_unstable_wasm_runtime() (SDK).
 - Node.js / Bun: the host engine's built-in WebAssembly API via N-API,
   adding zero dependency weight.
 - Python / Go / .NET / Java: Wasmtime compiled into the native binding.

This means the core can be built with no WASM dependency at all, and
JavaScript environments pay no overhead since they reuse V8 or JSC
natively.

    WASM instances are cached per-connection in an LRU keyed by module
name, with a shared atomic byte budget (default 10 MiB) across all
connections on a database. The cache avoids recompilation on repeated
calls while bounding total memory.

A Rust SDK crate (turso-wasm-sdk) provides a #[turso_wasm] proc macro
that transforms natural Rust functions into WASM exports with automatic
ABI marshalling for i64, f64, &str, &[u8], Option<T>, and return
types. A C shim (sqlite3_wasm_shim) bridges the standard sqlite3
extension API (sqlite3_create_function, sqlite3_value_*, sqlite3_result_*)
so that unmodified C extensions can be compiled to WASM with wasi-sdk
and loaded via CREATE EXTENSION.

Function and extension definitions are persisted in the schema
(turso_master) and re-registered on database open. DROP FUNCTION and
DROP EXTENSION clean up the schema, symbol table, and runtime modules.

The entire feature is marked unstable: CLI requires --unstable-wasm,
all SDK entry points use unstable-prefixed names, and documentation
carries experimental warnings. Only scalar functions are supported;
aggregate and window WASM functions are not yet available.

Preparation for the next commit which adds a non-Copy field
(Arc<dyn WasmRuntimeApi>) to DatabaseOpts. This is a mechanical
change that adds .clone() calls at all sites where DatabaseOpts
was previously implicitly copied. Split into its own commit to
keep the WASM feature commit focused on the actual feature.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@turso-bot turso-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please review @pedrocarlo

Introduce a system for extending Turso with custom scalar functions,
types, and virtual tables written in WebAssembly. Users register WASM
modules via two new SQL statements:

  CREATE FUNCTION name LANGUAGE wasm AS X'...' EXPORT 'export_name';
  CREATE EXTENSION name LANGUAGE wasm AS X'...';

Single functions (CREATE FUNCTION) export one callable with the
signature (argc: i32, argv: i32) -> i64, where arguments are
marshalled into the module's linear memory via a bump allocator
(turso_malloc). Extensions (CREATE EXTENSION) export a turso_ext_init
entry point that returns a JSON manifest declaring multiple functions,
custom types, and virtual tables, all registered in a single statement.
Unknown manifest fields are rejected so that modules targeting a newer
Turso version fail explicitly rather than silently losing resources.

The core design decision is a pluggable runtime: the database engine
defines a WasmRuntimeApi trait but bundles no WASM executor. Each host
environment injects its own:

 - CLI and Rust SDK: Wasmtime, with epoch-based interruption and fuel
   metering to bound runaway UDFs. Gated behind --unstable-wasm (CLI)
   and with_unstable_wasm_runtime() (SDK).
 - Node.js / Bun: the host engine's built-in WebAssembly API via N-API,
   adding zero dependency weight.
 - Python / Go / .NET / Java: Wasmtime compiled into the native binding.

This means the core can be built with no WASM dependency at all, and
JavaScript environments pay no overhead since they reuse V8 or JSC
natively.

WASM instances are cached per-connection in an LRU keyed by module
name, with a shared atomic byte budget (default 10 MiB) across all
connections on a database. The cache avoids recompilation on repeated
calls while bounding total memory.

A Rust SDK crate (turso-wasm-sdk) provides a #[turso_wasm] proc macro
that transforms natural Rust functions into WASM exports with automatic
ABI marshalling for i64, f64, &str, &[u8], Option<T>, and return
types. A C shim (sqlite3_wasm_shim) bridges the standard sqlite3
extension API (sqlite3_create_function, sqlite3_value_*, sqlite3_result_*)
so that unmodified C extensions can be compiled to WASM with wasi-sdk
and loaded via CREATE EXTENSION.

Function and extension definitions are persisted in the schema
(turso_master) and re-registered on database open. DROP FUNCTION and
DROP EXTENSION clean up the schema, symbol table, and runtime modules.

The entire feature is marked unstable: CLI requires --unstable-wasm,
all SDK entry points use unstable-prefixed names, and documentation
carries experimental warnings. Only scalar functions are supported;
aggregate and window WASM functions are not yet available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@penberg penberg changed the title Turso wasm udf WASM user-defined functions and extensions Apr 6, 2026
@tursodatabase tursodatabase deleted a comment from github-actions bot Apr 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant