Skip to content

README.md

Latest commit

 

History

History
159 lines (132 loc) · 8.06 KB

README.md

File metadata and controls

159 lines (132 loc) · 8.06 KB
Raw

An LLM on the Game Boy Advance

A 260,000-parameter language model (karpathy's stories260K) running on a 16.78 MHz ARM7TDMI with no FPU, 256KB of work RAM, and a 16-bit cartridge bus. It generates TinyStories-style text at ~2.4 tok/s (fresh context) to ~1.3 tok/s (context nearly full). Press A for a new story (reseeded from button timing), B to stop one early.

Built on a from-scratch bare-metal scaffold — no devkitPro/libgba, just arm-none-eabi-gcc and code you can read top to bottom.

Build & run

make        # produces build/game.gba (~294KB ROM)
make run    # builds and launches it in mGBA
make host   # native build of the same engine -> build/host_llm (fast testing)
make model  # regenerate model_data.c/h from model/*.bin (tools/export_model.py)

./build/host_llm [steps] [temperature] [seed] runs the exact engine code natively; temperature 0 is greedy decoding, which is what we diff against karpathy's float run.c to validate the fixed-point math.

How the LLM fits on a GBA

The ARM7TDMI has no FPU, and software floats would be brutally slow, so source/llm.c is a llama-2-style transformer written entirely in integer math (the same file compiles for host and GBA):

  • Weights (source/model_data.c, generated): int8, quantized per output row at export time, with Q24 fixed-point row scales. 259KB of ROM.
  • Activations: Q16.16 fixed point in int32. Each matmul dynamically quantizes its input vector to int8, does int8×int8→int32 dot products, then rescales via two 64-bit multiplies per output element.
  • KV cache: int8 with per-vector Q16 scales — 160KB, the main EWRAM tenant (total .bss is ~194KB of the 256KB).
  • Nonlinearities: RMSNorm uses a bitwise 64-bit integer sqrt; softmax and SiLU use a 256-entry exp2 LUT with interpolation; RoPE sin/cos come from Q14 tables precomputed at export.
  • Hot loops (matmul, attn_head, exp_q16, quantize) are compiled as ARM (not Thumb) and copied into IWRAM at boot (~1KB) — 32-bit bus, zero waitstates. REG_WAITCNT = 0x4317 (3/1 + prefetch) speeds up streaming the 259KB of weights from ROM every token.

Greedy decoding matches the float reference for the first ~17 tokens before tiny logit gaps flip a close argmax — story quality is indistinguishable.

Real-hardware checklist

Nothing left to do: make output boots on real hardware. Everything is hardware-honest (timers, DMA, waitstates, no emulator shortcuts), and the build's final step runs gbafix (vendored from devkitPro's gba-tools in tools/gbafix.c) to embed the Nintendo logo bitmap and fix the header checksums that the hardware BIOS verifies at boot. Copy build/game.gba to a flashcart and go.

How the GBA actually works

CPU. An ARM7TDMI (the same core used in a lot of late-90s phones), with two instruction sets: 32-bit ARM (more capable, bigger code) and 16-bit Thumb (denser, usually faster on this CPU because of its narrow memory bus). We build C code as Thumb and only the very first boot instruction as ARM, because the CPU always starts in ARM state.

Memory map. There's no OS, no filesystem, no malloc-by-default — just address ranges:

Region Address Size Notes
BIOS 0x00000000 16KB Nintendo's code; provides a few system calls (SWIs)
EWRAM 0x02000000 256KB "External" work RAM — general storage, slower bus
IWRAM 0x03000000 32KB "Internal" work RAM — on-chip, fast; stack lives here
I/O 0x04000000 - Memory-mapped hardware registers (video, input, sound, timers...)
PALRAM 0x05000000 1KB Color palettes
VRAM 0x06000000 96KB Video RAM — tiles, bitmaps, backgrounds
OAM 0x07000000 1KB Object Attribute Memory — sprite positions/state
ROM 0x08000000 ≤32MB The cartridge — where your code and assets live

There's no "print to console" or "malloc" — you make things happen by writing to specific addresses in the I/O region. include/gba.h names the few we use so far (REG_DISPCNT, REG_VCOUNT, REG_KEYINPUT). Add more as needed — the GBA has dozens (background scroll registers, sprite/OAM control, sound channels, timers, DMA, interrupts).

Boot sequence (source/crt0.s). The BIOS jumps to 0x08000000 in ARM mode and expects, in order: one branch instruction, then a fixed 192-byte header (a Nintendo logo bitmap it checksums, plus title/game code/maker bytes). crt0.s builds that header, sets up a stack pointer for IRQ and System CPU modes (the ARM7TDMI banks registers per mode), copies initialized globals (.data) from ROM into EWRAM, zeroes uninitialized globals (.bss), then jumps into main().

The Nintendo logo bytes are zeroed in crt0.s; the build's final gbafix step (tools/gbafix.c, vendored from devkitPro) patches the real bitmap and header checksums into build/game.gba, so the shipped ROM boots on real hardware. mGBA doesn't verify the logo, but the hardware BIOS does.

Linker script (gba.ld). Tells the linker to place code/read-only data in ROM, and put initialized/uninitialized globals in EWRAM. This is also where the symbols crt0.s uses (__data_start, __bss_end, etc.) come from.

Video. main.c uses "Mode 3": a single 240×160 framebuffer, one 16-bit color per pixel (5 bits each for R/G/B), written directly into VRAM — simplest mode to reason about, closest to "just draw pixels." Real GBA games mostly use Mode 0: tiled backgrounds (reusable 8×8 tile graphics referencing a palette, arranged on a tilemap) plus hardware sprites (independently positioned objects drawn from OAM), because tiles are dramatically cheaper on VRAM and CPU than redrawing a full bitmap every frame. Once you have sprite/tile art, that's the next thing to add here — happy to build the Mode 0 + sprite path when you have assets to test it against.

Timing. The screen draws one scanline at a time, 228 lines per frame (160 visible + 68 vblank), giving a ~59.7Hz frame rate. main.c polls REG_VCOUNT to detect vblank (the safe window to update VRAM without tearing) by busy-waiting. The more efficient real-world approach is to enable the vblank hardware interrupt and call the BIOS VBlankIntrWait SWI, which lets the CPU idle instead of spinning — worth adding once the game loop needs the CPU time back.

Input. REG_KEYINPUT is inverted (0 = pressed) for historical reasons; key_poll() in gba.h flips it so KEY_LEFT etc. read naturally.

Adding your art/audio later

  • Tiles/sprites/backgrounds: convert PNGs to GBA tile format with grit (part of devkitPro, or usable standalone) — outputs .c/.h arrays you #include and DMA into VRAM/palette memory.
  • Audio: the GBA has no built-in synth worth using raw; real projects use a tracker/driver (Maxmod is standard with devkitPro). We haven't wired anything audio-related yet — the register-poking approach doesn't scale well here, so this is a good place to actually pull in a library.
  • If any of this gets tedious in raw C, Butano (a C++ engine on top of devkitARM) wraps sprites/backgrounds/audio/scenes in a much friendlier API, at the cost of not seeing the registers directly. Worth reconsidering once the hardware model above feels familiar.

Project layout

source/crt0.s     boot code, CPU mode setup, ROM header, IWRAM/.data copy
source/main.c     generation loop, status line, mGBA debug logging
source/llm.c      the integer-only transformer engine (host + GBA)
source/console.c  mode 3 text console (font8x8, DMA scrolling)
source/model_data.c   generated: int8 weights, tables, tokenizer (make model)
include/gba.h     hardware register definitions
include/llm.h     engine API + IWRAM placement of hot code
host/test_llm.c   native test harness for the engine
model/            stories260K.bin + tok512.bin (fp32 originals)
tools/export_model.py quantizes the model into source/model_data.c
gba.ld            linker script (ROM/EWRAM/IWRAM layout)
tools/gbafix.c    devkitPro's header fixer (logo + checksums), run by make