feat: Windows FFI build tooling + livekit_ffi.dll/pdb with symbols

.gitattributes

@@ -1,5 +1,6 @@
 *.a filter=lfs diff=lfs merge=lfs -text
 *.dll filter=lfs diff=lfs merge=lfs -text
+*.pdb filter=lfs diff=lfs merge=lfs -text
 *.so filter=lfs diff=lfs merge=lfs -text
 *.dylib filter=lfs diff=lfs merge=lfs -text
 Runtime/Plugins/arm64/liblivekit_ffi.dylib filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -1,2 +1,6 @@
 /downloads~/*
 .DS_Store
+
+# Windows FFI build tool: downloaded source + binaries staged in the tool folder
+/BuildScripts~/windows/.src/
+/BuildScripts~/windows/livekit_ffi.*

BuildScripts~/windows/BUILD-WINDOWS.md

@@ -0,0 +1,86 @@
+# Building `livekit_ffi.dll` + `livekit_ffi.pdb` on Windows (x86_64)
+
+> **Build tooling, not a Unity asset.** These scripts live under `BuildScripts~/` (the trailing
+> `~` keeps the folder out of Unity's asset pipeline). They **download** the rust source into a
+> gitignored `.src/` subfolder and build the native FFI library there; the source is never committed.
+> They are **not** part of the shipped package and are not run from `Runtime/Plugins`.
+
+Build the **`livekit-ffi`** crate as a release cdylib **with PDB debug symbols** on Windows, target `x86_64-pc-windows-msvc`. Everything is driven by `build.config.toml`:
+
+| File | What it is |
+|------|------------|
+| `build.config.toml` | Config: `Tag` (which release to download/build) and `InstallToPlugins` (where the output goes). Edit this, not the scripts. |
+| `Patch.toml` | The `[profile.release]` overlaid onto the downloaded `Cargo.toml`. See its header comment for what each flag does and why; edit it to change build flags. |
+| `env-setup.ps1` | One-time bootstrap (PowerShell, so it runs on a bare machine): installs Python 3, VS2022 Build Tools (MSVC v143) + Windows 11 SDK, Git, Rust (MSVC), libclang, protoc, dasel. |
+| `build-win.py` | Clones the `Tag` source (with nested submodules) into `.src/`, overlays `Patch.toml` onto `[profile.release]` for PDB output, runs `cargo build`, and places the DLL + PDB per `InstallToPlugins`. |
+
+## Quick start
+
+```powershell
+cd BuildScripts~/windows
+.\env-setup.ps1             # one-time bootstrap (installs Python + the toolchain)
+notepad build.config.toml   # set Tag + InstallToPlugins
+python build-win.py
+```
+
+## Configuration (`build.config.toml`)
+
+```toml
+Tag                   = "livekit-ffi/v0.12.48"   # release to build; tags at the releases page below
+SourceDir             = ".src"                   # where to clone the source
+InstallToPlugins      = true                     # see below
+CleanSourceAfterBuild = false                    # delete the clone after a successful build
+```
+
+- `Tag`: a `livekit-ffi/vX.Y.Z` tag from <https://github.com/livekit/rust-sdks/releases>.
+- `SourceDir`: where the source is cloned. Relative paths resolve against this tool folder; absolute paths (e.g. `C:/src`) work too. The default `.src` is gitignored; if you point it at another in-repo folder, add that to `.gitignore` yourself. An absolute path outside the repo also sidesteps the long-paths note below.
+- `InstallToPlugins`:
+  - `true` -> the built `livekit_ffi.dll` + `livekit_ffi.pdb` **replace** the ones in `Runtime/Plugins/ffi-windows-x86_64/` (ready to ship).
+  - `false` -> they are dropped in this tool folder (`BuildScripts~/windows/`, gitignored) for inspection; copy them over yourself.
+- `CleanSourceAfterBuild`: `true` removes the `<SourceDir>/rust-sdks-<tag>` checkout once the DLL + PDB are placed; `false` (default) keeps it so re-runs skip the clone.
+
+The source is cloned to `<SourceDir>\rust-sdks-<tag>\` and reused on re-runs. The two output files:
+
+| File | Purpose |
+|------|---------|
+| `livekit_ffi.dll` | Runtime cdylib. CRT is statically linked (`+crt-static`), so no VC++ Redistributable is needed. |
+| `livekit_ffi.pdb` | Debug symbols, paired to that exact DLL (matching RSDS GUID). Needed only to debug/symbolicate, not to run. |
+
+## Notes on the build
+
+- **Source is downloaded, not vendored.** `build-win.py` does `git clone --recurse-submodules` of `livekit/rust-sdks` at the tag into `.src/` (gitignored); this also pulls the nested `yuv-sys/libyuv` + `livekit-protocol/protocol` submodules. webrtc is downloaded separately by `livekit-ffi/build.rs`. Nothing is added to this repo. (The `client-sdk-rust~` submodule that lives here is for C# proto generation via `generate_proto.sh`, not for this build.)
+- **`+crt-static` comes from upstream, not the script.** The downloaded source's `.cargo/config.toml` sets `target-feature=+crt-static` for `x86_64-pc-windows-msvc`; cargo picks it up because the build runs from the source root. The script does not set it.
+- **The profile patch is deliberate, and done with dasel.** `build-win.py` overlays `Patch.toml`'s `[profile.release]` onto the downloaded `Cargo.toml` using dasel, feeding the source `Cargo.toml` to dasel on stdin; see `Patch.toml` for the flags and what each is for. dasel reformats the file (reorders tables, single-quotes strings, drops comments), which is harmless since the checkout under `.src/` is a throwaway. This matches upstream's own release profile, so for recent tags it just re-asserts existing values. It requires the source to already define `[profile.release]` (livekit-ffi tags do); if one ever doesn't, the patch step fails loudly rather than silently shipping a symbol-less build.
+- **Long paths (read this if the C++ compile fails).** webrtc's bundled headers nest deeply: under the default `SourceDir = ".src"` (inside this repo) their full paths reach ~390 characters, well past the Windows 260-char `MAX_PATH` limit. The clone itself survives (git runs with `core.longpaths=true`), but `cl.exe` is not long-path aware unless the machine has `LongPathsEnabled=1`, so the webrtc-sys C++ compile fails with `fatal error C1083: Cannot open include file` **on a header that actually exists** - the path is simply too long to open. Two fixes, either is enough:
+  - Set `SourceDir` to a short absolute path **outside** the repo, e.g. `C:/src` (recommended - no admin needed). This is what shortens the deep paths back under the limit.
+  - Or enable Windows long paths once, as admin: set `HKLM\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled = 1` (DWORD) and reboot.
+
+## Why VS2022 + Windows 11 SDK
+
+The prebuilt `webrtc.lib` references VS2022 STL symbols (`__std_find_trivial_*`) and is built against the Windows 11 SDK (`NTDDI_WIN11_*`). VS2019 or a Windows 10 SDK will not compile or link.
+
+## Shipping notes
+
+- To **run**: ship `livekit_ffi.dll` only. It loads on stock Windows 10/11 **x86_64** in any 64-bit host process.
+- To **debug/symbolicate**: keep `livekit_ffi.pdb` paired with that exact DLL, or put it in a symbol store.
+- Architecture: **x86_64** only. For arm64 build `aarch64-pc-windows-msvc` separately; a 64-bit DLL cannot load into a 32-bit process.
+
+## Troubleshooting (symptom → cause)
+
+| Symptom | Cause / fix |
+|---|---|
+| `Python installed but not yet on PATH` (env-setup) | env-setup just installed Python; open a new terminal so PATH refreshes, then re-run `.\env-setup.ps1`. |
+| `python` is not recognized (running build-win.py) | Python not on PATH yet; open a new terminal after `env-setup.ps1`, or call it by full path. |
+| `git not found` (build-win) | Git missing; re-run `env-setup.ps1` (it installs Git). |
+| `dasel not found` (build-win) | dasel missing; re-run `env-setup.ps1` (it installs dasel). |
+| `dasel failed to patch [profile.release]` | The chosen `Tag` has no `[profile.release]`; add one to `Patch.toml`'s target or pick a tag that defines it. |
+| `yuv-sys` build.rs panics `NotFound` reading `include/libyuv` | libyuv submodule not fetched; delete the `.src\rust-sdks-*` checkout and re-run so the clone pulls submodules. |
+| webrtc-sys C++: `fatal error C1083: Cannot open include file` (header that exists) | Paths exceed the 260-char `MAX_PATH` limit. Set `SourceDir` to a short path like `C:/src`, or enable `LongPathsEnabled=1` (admin). See the long-paths note above. |
+| `bindgen`: `Unable to find libclang` | `LIBCLANG_PATH` unset/wrong → re-run `env-setup.ps1`. |
+| build.rs: `Could not find protoc` | `PROTOC` unset → re-run `env-setup.ps1`. |
+| `fileapi.h: error C2061: ... 'FILE_INFO_BY_HANDLE_CLASS'` | SDK too old; `NTDDI_WIN11_*` undefined → install a Windows 11 SDK. |
+| `LNK2019/LNK2001: __std_find_trivial_*`, `__std_find_last_trivial_*` | Linking with VS2019 STL; build with the VS2022 v143 toolset. |
+
+## Manual build
+
+If you prefer not to use the scripts, see `build-win.py` for the exact steps: install the toolchain, `git clone --recurse-submodules` the tag, set the `[profile.release]` block (the contents of `Patch.toml`), then run `cargo build --release -p livekit-ffi` from the source root in a shell with `vcvarsall.bat x64 <SDKVER>` loaded and `LIBCLANG_PATH` / `PROTOC` set.

BuildScripts~/windows/Patch.toml

@@ -0,0 +1,14 @@
+# Patch.toml - the [profile.release] that build-win.py overlays onto the downloaded Cargo.toml,
+# so the build emits separate debug symbols (a .pdb on Windows, a .dSYM on macOS) instead of
+# baking them into the library. This mirrors upstream rust-sdks' own release profile; it is the
+# single source for these flags - edit here to change them.
+[profile.release]
+# Emit debug symbols to a separate file (.pdb on Windows, .dSYM on macOS) instead of the binary:
+debug = 2
+split-debuginfo = "packed"
+strip = "symbols"
+# Optimization flags below match upstream's release profile.
+lto = true
+opt-level = "z"
+codegen-units = 1
+panic = "abort"

BuildScripts~/windows/build-win.py

@@ -0,0 +1,214 @@
+#!/usr/bin/env python3
+"""build-win.py - build livekit_ffi.dll + livekit_ffi.pdb (release, x86_64-pc-windows-msvc).
+
+Self-contained: reads build.config.toml, clones the rust-sdks source into .src/ (gitignored,
+never committed), patches [profile.release] for PDB output, runs cargo build, and places the
+DLL + PDB per the config flag. Run env-setup.ps1 first to install the toolchain (incl. Python).
+"""
+
+import os
+import shutil
+import subprocess
+import sys
+import tomllib
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+REPO_ROOT = HERE.parents[1]
+PROTOC_DIR = Path(os.environ.get("LOCALAPPDATA", "")) / "protoc"  # where env-setup.ps1 put protoc
+DASEL_DIR = Path(os.environ.get("LOCALAPPDATA", "")) / "dasel"    # where env-setup.ps1 put dasel
+
+
+def step(msg):
+    print(f"\n==> {msg}")
+
+
+def info(msg):
+    print(f"    {msg}")
+
+
+def die(msg):
+    sys.exit(f"ERROR: {msg}")
+
+
+def run(args, cwd=None, env=None, fail=None):
+    """Run a command, streaming its output; exit with `fail` on a nonzero code."""
+    code = subprocess.run([str(a) for a in args], cwd=cwd, env=env).returncode
+    if code != 0:
+        die(f"{fail or 'command failed'} (exit {code})")
+
+
+def find_tool(name, fallback):
+    """A tool on PATH, else the copy env-setup.ps1 dropped in LOCALAPPDATA."""
+    found = shutil.which(name)
+    return Path(found) if found else Path(fallback)
+
+
+def vcvars_env(vcvars, sdk_ver):
+    """Source vcvarsall.bat and return the environment it sets.
+
+    vcvarsall.bat is a batch script, so importing its variables means running it under cmd and
+    dumping `set`; there is no native alternative. This is the build env only - unrelated to dasel.
+    """
+    res = subprocess.run(f'"{vcvars}" x64 {sdk_ver} >nul 2>&1 && set',
+                         shell=True, capture_output=True, text=True)
+    if res.returncode != 0:
+        die(f"vcvarsall.bat failed: {res.stderr.strip()}")
+    env = dict(os.environ)
+    for line in res.stdout.splitlines():
+        key, sep, val = line.partition("=")
+        if sep:
+            env[key] = val
+    return env
+
+
+def main():
+    # --- 0. Config ---------------------------------------------------------
+    cfg_path = HERE / "build.config.toml"
+    if not cfg_path.is_file():
+        die(f"config not found: {cfg_path}")
+    cfg = tomllib.loads(cfg_path.read_text(encoding="utf-8"))
+    tag = cfg.get("Tag")
+    if not tag:
+        die("Tag not set in build.config.toml")
+    install_to_plugins = bool(cfg.get("InstallToPlugins", False))
+    clean_source = bool(cfg.get("CleanSourceAfterBuild", False))
+
+    src_cfg = cfg.get("SourceDir") or ".src"                       # clone root from config
+    src_root = Path(src_cfg) if Path(src_cfg).is_absolute() else HERE / src_cfg
+    repo = src_root / ("rust-sdks-" + tag.replace("/", "-").replace("\\", "-"))
+
+    # --- 1. Ensure toolchain ----------------------------------------------
+    step("Ensure toolchain")
+    git = shutil.which("git")
+    if not git:
+        die("git not found - run env-setup.ps1 first.")
+    cargo = find_tool("cargo", Path(os.environ["USERPROFILE"]) / ".cargo" / "bin" / "cargo.exe")
+    if not cargo.exists():
+        die("cargo not found - run env-setup.ps1 first.")
+    # dasel patches Cargo.toml's [profile.release]. Prefer one on PATH, else env-setup's copy.
+    dasel = find_tool("dasel", DASEL_DIR / "dasel.exe")
+    if not dasel.exists():
+        die("dasel not found - run env-setup.ps1 first.")
+
+    # Locate VS2022 + vcvarsall via vswhere.
+    vswhere = Path(os.environ["ProgramFiles(x86)"]) / "Microsoft Visual Studio" / "Installer" / "vswhere.exe"
+    vs_path = subprocess.run(
+        [str(vswhere), "-latest", "-products", "*",
+         "-requires", "Microsoft.VisualStudio.Component.VC.Tools.x86.x64",
+         "-property", "installationPath"],
+        capture_output=True, text=True).stdout.strip()
+    if not vs_path:
+        die("VS2022 C++ tools not found - run env-setup.ps1 first.")
+    vcvars = Path(vs_path) / "VC" / "Auxiliary" / "Build" / "vcvarsall.bat"
+
+    # Newest installed Windows SDK (build against the Win11 SDK headers).
+    sdk_inc = Path(os.environ["ProgramFiles(x86)"]) / "Windows Kits" / "10" / "Include"
+    sdks = sorted(p.name for p in sdk_inc.iterdir() if p.is_dir() and p.name.startswith("10.")) if sdk_inc.is_dir() else []
+    if not sdks:
+        die("no Windows 10/11 SDK found - run env-setup.ps1 first.")
+    sdk_ver = sdks[-1]
+
+    # libclang + protoc: env vars consumed by the rust build scripts. We are already in Python,
+    # so locate libclang by importing the clang package instead of shelling back out to it.
+    try:
+        import clang
+        libclang_path = Path(clang.__file__).parent / "native"
+    except ImportError:
+        die("libclang (python 'clang' package) not found - run env-setup.ps1 first.")
+    protoc = PROTOC_DIR / "bin" / "protoc.exe"
+    if not libclang_path.exists():
+        die("libclang not found - run env-setup.ps1 first.")
+    if not protoc.exists():
+        die("protoc not found - run env-setup.ps1 first.")
+
+    info(f"tag:           {tag}")
+    info(f"source:        {repo}")
+    info(f"install:       {'Runtime/Plugins/ffi-windows-x86_64' if install_to_plugins else 'tool folder'}")
+    info(f"cargo:         {cargo}")
+    info(f"vcvars:        {vcvars}")
+    info(f"WinSDK:        {sdk_ver}")
+    info(f"LIBCLANG_PATH: {libclang_path}")
+    info(f"PROTOC:        {protoc}")
+    info(f"dasel:         {dasel}")
+
+    # --- 2. Download source for the tag (into .src/, with nested submodules) ---
+    # git clone --recurse-submodules pulls yuv-sys/libyuv + livekit-protocol/protocol cleanly;
+    # webrtc is downloaded later by livekit-ffi/build.rs. Per-tag folder, reused on re-run.
+    # core.longpaths handles webrtc's deep paths nested under this repo folder.
+    step(f"Download rust-sdks source ({tag})")
+    src_root.mkdir(parents=True, exist_ok=True)
+    if (repo / ".git").is_dir():
+        info(f"reusing existing checkout: {repo}")
+    else:
+        if repo.exists():
+            shutil.rmtree(repo)                                    # clear a partial/failed checkout
+        run([git, "-c", "core.longpaths=true", "clone", "--depth", "1", "--branch", tag,
+             "--recurse-submodules", "--shallow-submodules",
+             "https://github.com/livekit/rust-sdks.git", repo],
+            fail=f"git clone failed for tag {tag}")
+        info(f"source at: {repo}")
+    libyuv = repo / "yuv-sys" / "libyuv" / "include" / "libyuv"
+    if not libyuv.exists():
+        run([git, "-C", repo, "submodule", "update", "--init", "--recursive"])
+    if not libyuv.exists():
+        die("libyuv submodule not populated")
+
+    # --- 3. Patch [profile.release] (traceable build with PDB) -------------
+    # Overlay Patch.toml's [profile.release] onto the downloaded Cargo.toml with dasel (see Patch.toml
+    # for what each flag does and why). dasel rewrites Cargo.toml (reorders/quotes/drops comments), but
+    # .src/ is a throwaway clone. dasel reads the source toml from stdin; we hand it the open file so
+    # no shell redirect is needed.
+    step("Patch [profile.release]")
+    cargo_toml = repo / "Cargo.toml"
+    patch_toml = HERE / "Patch.toml"
+    with open(cargo_toml, "rb") as stdin_file:
+        res = subprocess.run(
+            [str(dasel), "-i", "toml", "-o", "toml",
+             "--var", f"patch=toml:file:{patch_toml}",
+             "profile.release = $patch.profile.release", "--root"],
+            stdin=stdin_file, capture_output=True)
+    if res.returncode != 0 or not res.stdout.strip():
+        die(f"dasel failed to patch [profile.release] (does {tag} define one?): "
+            f"{res.stderr.decode('utf-8', 'replace').strip()}")
+    # Write back the dasel output verbatim (already UTF-8, no BOM), with one trailing LF.
+    cargo_toml.write_bytes(res.stdout.rstrip() + b"\n")
+    info(f"patched {cargo_toml} via dasel")
+
+    # --- 4. Build ----------------------------------------------------------
+    step("cargo build --release -p livekit-ffi")
+    env = vcvars_env(vcvars, sdk_ver)
+    env["LIBCLANG_PATH"] = str(libclang_path)
+    env["PROTOC"] = str(protoc)
+    env["PATH"] = str(Path(os.environ["USERPROFILE"]) / ".cargo" / "bin") + os.pathsep + env.get("PATH", "")
+    run([cargo, "build", "--release", "-p", "livekit-ffi"], cwd=repo, env=env, fail="cargo build failed")
+
+    # --- 5. Place output ---------------------------------------------------
+    rel = repo / "target" / "release"
+    dll, pdb = rel / "livekit_ffi.dll", rel / "livekit_ffi.pdb"
+    if not dll.exists():
+        die(f"build produced no DLL at {dll}")
+    if not pdb.exists():
+        die(f"build produced no PDB at {pdb}")
+    dest = (REPO_ROOT / "Runtime" / "Plugins" / "ffi-windows-x86_64") if install_to_plugins else HERE
+    dest.mkdir(parents=True, exist_ok=True)
+    shutil.copy2(dll, dest)
+    shutil.copy2(pdb, dest)
+
+    # --- 6. Clean source (optional) ----------------------------------------
+    if clean_source:
+        step("Clean source")
+        shutil.rmtree(repo, ignore_errors=True)
+        info(f"removed {repo}")
+
+    step("Done")
+    print(f"    DLL -> {dest / 'livekit_ffi.dll'}")
+    print(f"    PDB -> {dest / 'livekit_ffi.pdb'}")
+    if not install_to_plugins:
+        print("    (InstallToPlugins is false; copy these into Runtime/Plugins/ffi-windows-x86_64/ to ship.)")
+
+
+if __name__ == "__main__":
+    if os.name != "nt":
+        die("this build targets x86_64-pc-windows-msvc and must run on Windows.")
+    main()