Metal: GGML_ASSERT(buft) crash in ggml_metal_graph_optimize_reorder when creating context on Apple M5 (prebuilt b9842)

[TonyWang-hub]

Issue description

Creating a context crashes the process on an Apple M5 MacBook Air with the
prebuilt Metal binaries shipped in v3.19.0 (llama.cpp build b9842).
Metal shader compilation fails first, then a GGML assert aborts the process.

Possibly related to #549 / #595 (other M5 Metal problems), but the crash
signature is different.

Environment

• MacBook Air, Apple M5 (10-core: 4P+6E), 16 GB unified memory
• macOS 26
• node-llama-cpp 3.19.0 (prebuilt Metal binaries, libllama.metal.b9842.dylib)
• Reproduces under both Node.js 22.22.3 and Bun

Error output

[node-llama-cpp] ggml_metal_library_init_from_source: error compiling source
/Users/runner/work/node-llama-cpp/node-llama-cpp/llama/llama.cpp/ggml/src/ggml-backend.cpp:62: GGML_ASSERT(buft) failed
WARNING: Using native backtrace. Set GGML_BACKTRACE_LLDB for more info.
0   libggml-base.dylib            ggml_print_backtrace + 276
1   libggml-base.dylib            ggml_abort + 156
2   libggml-base.dylib            ggml_backend_buft_is_host + 0
3   libggml-metal.so              ggml_metal_graph_optimize_reorder (lambda) + 364
4   libggml-metal.so              ggml_graph_optimize + 2268
5   libggml-base.dylib            ggml_backend_sched_split_graph + 6804
6   libllama.metal.b9842.dylib    llama_context::graph_reserve + 496
7   libllama.metal.b9842.dylib    llama_context::sched_reserve + 2184
8   libllama.metal.b9842.dylib    llama_context::llama_context + 4476
9   libllama.metal.b9842.dylib    llama_init_from_model + 588
10  llama-addon.node              AddonContextLoadContextWorker::Execute + 108
11  llama-addon.node              Napi::AsyncWorker::OnExecute + 32
12  bun                           bun + 10080272
13  bun                           bun + 11565400
14  libsystem_pthread.dylib       _pthread_start + 136
15  libsystem_pthread.dylib       thread_start + 8

(Note: the /Users/runner/... path above is from the CI machine that built
the prebuilt binaries, not my machine.)

Repro

Any model.createContext() with GPU enabled crashes this way on M5.
The same model and code work fine with the GPU disabled.

Verified workaround

NODE_LLAMA_CPP_GPU=false + a one-time local source build (cmake required;
there is no prebuilt CPU-only binary for mac-arm64). After that, model load
and context creation work under both Node and Bun.

Happy to run diagnostics on this M5 machine if it helps — I saw in #549 that
M5 hardware is hard to come by for testing.

[giladgd]

@TonyWang-hub which model did you try to run?
Could you please provide a simple contained reproduction that I can run?
Also please include the result of running npx -y node-llama-cpp inspect gpu.
I’ve got an M5 Max machine to test on.

[TonyWang-hub]

Model: hf:tobil/qmd-query-expansion-1.7B-gguf/qmd-query-expansion-1.7B-q4_k_m.gguf (a small 1.7B GGUF used by clawmem, the tool that triggers this).

npx -y node-llama-cpp inspect gpu output:

OS: macOS 25.5.0 (arm64)
Node: 22.22.3 (arm64)
node-llama-cpp: 3.19.0
Prebuilt binaries: b9842

Metal: available
Metal device: Apple M5
Metal free VRAM: 99.99% (11.84GB/11.84GB)

ggml_metal_library_init_from_source: error compiling source
ggml_metal_device_init: - the tensor API is not supported in this environment - disabling
CPU model: Apple M5
Math cores: 4

This reproduces every time I run inspect gpu — the error compiling source line shows up consistently.

Re: a minimal repro — I put together a bare script that just does getLlama() → loadModel() → createContext() with the same model. Interestingly it's not deterministic: most runs (including 3 concurrent runs of the same script) succeed — I can see the same error compiling source warning in the debug log, but node-llama-cpp recovers and falls back to MTLGPUFamilyMetal4 successfully. The original crash from my first report only happens sometimes, when that fallback recovery itself fails and hits GGML_ASSERT(buft).

So: inspect gpu looks like the cleanest single-command way to see the underlying Metal shader compile issue on this hardware, even though it doesn't crash by itself. The full context-creation crash is intermittent on top of that. Happy to run a repro script in a loop to get you a failure rate, or run anything else you'd like tested on this M5.

[giladgd]

@TonyWang-hub Could you please run the following commands and share the output with me?
It'll enable more verbose logging around the area that prints the ggml_metal_library_init_from_source: error compiling source error so we can see more info about it.

npx --no node-llama-cpp source download --skipBuild
find ./node_modules/node-llama-cpp/llama/llama.cpp -type f -name 'ggml-metal-device.m' -exec perl -pi -e 's/\Qif (verbose) {\E/if (true) {/g' {} +
npx --no node-llama-cpp source build --gpu metal

npx --no node-llama-cpp inspect gpu