March updates

I need to move some work over from the private repos. The exploration into using massivly multheaded models with mixer and readout functions separated continues. Currently training up a Chinese TinyStories on a 3B-equivalent model that trains, easily, on an 8GB gpu. We're looking at 80-90% reduction in needed GPU memory. Will move more of the work over this coming week.

January Updates

It was a fun holiday. First IPs have been filed. Several POCs done with all-optical inference (no GPU!) to explore the envelope of what is possible:

1. AlphaGo, a unitary ish Born-rule collapse all-optical Go player (this is from one overnight training)

https://github.com/dwallener/EntropicaPublic/blob/main/v0.4-alphago/README.md

2. 2gps, similar optical pipeline figuring out its coordinates, altitude and heading based on what it sees out the forward view camera

• Two versions: straight HUD-like output and (more, better) encoding the coordinates into a form suitable for secure line-of-sight laser comms

https://github.com/dwallener/EntropicaPublic/blob/main/v0.4-gps/README.md

3. Target, this time we detect and track targets

https://github.com/dwallener/EntropicaPublic/blob/main/v0.4-target/README.md

Background (WIP)

We present Entropica, the first generative language model whose forward pass is physically realizable as a passive linear-optical interferometer operating at zero electrical power during inference.

The model uses a 1024-dimensional complex Hilbert space with 32 layers of programmable Mach–Zehnder meshes (Reck architecture) and derives token probabilities directly via the Born rule on a 650 nm laser beam.

Despite using only unitary operations and no attention mechanism, a 1024×32 model achieves coherent TinyStories generation after < 1.8 hours of training on a single consumer GPU. We further demonstrate a complete optical implementation path using printed phase masks on transparency film and a $30 laser diode.

Most recent technical note:

https://zenodo.org/records/17764289

Public Facing Repo for Quantum-LM Exploration

v001

https://zenodo.org/records/17764289

The source code for the first Entropica paper. The model itself is in v000/quantum_lm.py

https://github.com/dwallener/EntropicaPublic/blob/main/v001/quantum_lm.py

The scripts generate/sample/train manage the dataset, inference and training process.