local-llm

Run Claude Code against a local model via llama.cpp.

Requirements

• llama-server (from llama.cpp, installable via brew install llama.cpp)
• jq (installable via brew install jq)
• claude CLI

Usage

1. Start the server

# Recommended preset (Gemma 4 26B, 64k ctx, thinking on by default)
start-llama --preset gemma4-26b

# Different context size (aliases or raw integer)
start-llama --preset gemma4-26b 128k
start-llama --preset gemma4-26b 200000

# Disable reasoning mode
start-llama --preset gemma4-26b --no-thinking

# Any HuggingFace GGUF model
start-llama <hf-repo:file> [ctx-size]
start-llama ggml-org/gemma-4-E4B-it-GGUF:Q4_K_M 32k

Context size aliases: 8k, 16k, 32k, 64k, 128k, 256k, 512k — or any integer.

2. Run Claude Code

cc-local
cc-local --resume

cc-local auto-detects the running model from the server. Any additional arguments are forwarded to the claude CLI.

Shell setup

Add to your .zshrc:

alias start-llama="~/local-llm/start-llama.sh"
source ~/local-llm/shell/cc-local.sh

References

This repo was built using the following resources:

• Running Claude Code with local LLMs
• Running Google Gemma 4 locally with llama.cpp