S3 Increment 2: fan out the visual surface (learn-loop + rollout GIFs, MAP-Elites illumination)

Reuse Increment 1's Mojo-trace -> dependency-free renderer -> committed-asset
pipeline for the three artifacts named in the S3 plan. Everything stays numpy +
stdlib (no Pillow): a self-contained GIF89a/LZW encoder in tools/viz/gifpng.py,
validated by a full encode->decode round-trip over every frame.

- "Watch it learn" (flip_h): src/main.mojo gains --learn-trace; the capture is a
  viz-only twin forward_with_learning_traced in esper_evolution.mojo that mirrors
  fit_operator's annealed schedule exactly and snapshots the held-out prediction
  into a pre-allocated buffer BETWEEN ES steps (never in the hot loop). Renderer:
  tools/viz/render_learn.py (GIF + filmstrip).
- Repertoire rollout: src/sandbox_trace.mojo loads a .rep, picks the
  highest-occupancy elite, and rolls it out via a new viz-only
  sandbox_rollout_trace twin in sandbox.mojo that snapshots the grid per tick;
  the ES-critical rollouts are untouched. Renderer: tools/viz/render_rollout.py.
- MAP-Elites illumination (zero Mojo change): tools/viz/render_repertoire.py
  reads the existing .rep binary directly (avatar-position coverage + paint field).

Production forward_with_learning / fit_operator / ES rollouts are byte-untouched;
the additions are strictly additive/flag-gated. ./esper fast stays green (3/3
held-out). README gains a "Watch it work" section + reproduce blocks; JOURNAL
records the entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: tacio

README.md

@@ -29,6 +29,29 @@ Under the hood is **Nested Learning** (the HOPE architecture):
 The engine learns the transformation; it is never handed a symbolic DSL. Geometric,
 colour, and count transforms *emerge* as fitted operator parameters.
 
+## Watch it work
+
+**Learning a transform in-context.** The operator starts at the identity prior and
+is fitted by the ES on the demo pairs. Below is its prediction on the **held-out**
+input converging over 4 000 iterations — from the identity, through noise, to an
+exact `flip_h`:
+
+![Esper's held-out prediction converging from identity to an exact horizontal flip over ES iterations.](docs/assets/learn.gif)
+
+**A learned open-ended behaviour.** Esper's reward-free MAP-Elites repertoire
+illuminates a space of sandbox behaviours through the *unchanged* ES core. One
+elite's policy, rolled out under the sandbox's gravity, paints the world floor-up:
+
+![A repertoire policy painting the sandbox world, its paint settling under gravity over 64 ticks.](docs/assets/rollout.gif)
+
+…and the repertoire's coverage of that behaviour space — the avatar-final-position
+illumination map (left) and the mean paint field (right):
+
+![MAP-Elites illumination: coverage of avatar-position behaviour space, and the repertoire's mean paint field.](docs/assets/repertoire.png)
+
+Every frame is rendered offline from a `--trace` / `.rep` dump by a dependency-free
+renderer (numpy + stdlib only) — nothing is hand-drawn. See [Reproduce](#reproduce).
+
 ## Results (honest, including the negatives)
 
 The evidence trail is the currency here. Everything is fit on train pairs and scored
@@ -90,6 +113,22 @@ python tools/viz/render_hero.py /tmp/trace.jsonl --out docs/assets/hero.png
 pred, true, …}`) *after* each fit — never inside the ES hot loop. The renderer is
 dependency-free (numpy + stdlib `zlib` only).
 
+**The "watch it learn" animation** (per-K-iter held-out prediction → GIF + filmstrip):
+```bash
+./esper run src/main.mojo --learn-trace /tmp/learn.jsonl
+python tools/viz/render_learn.py /tmp/learn.jsonl   # -> docs/assets/learn.{gif,png}
+```
+
+**A repertoire rollout + illumination map** (needs a `.rep` under `build/`, produced
+by the repertoire tests or a MAP-Elites run — `build/` is gitignored):
+```bash
+./esper run src/sandbox_trace.mojo build/b_poc_4.rep /tmp/rollout.jsonl
+python tools/viz/render_rollout.py /tmp/rollout.jsonl      # -> docs/assets/rollout.{gif,png}
+python tools/viz/render_repertoire.py build/b_poc_4.rep    # -> docs/assets/repertoire.png
+```
+The animated GIFs come from a self-contained GIF89a/LZW encoder in
+`tools/viz/gifpng.py` — still numpy + stdlib only, no Pillow.
+
 **The interactive triptych** (marimo + anywidget, decoupled from the engine):
 ```bash
 uv pip install marimo anywidget      # viz-only deps; the engine needs none of them

docs/JOURNAL.md

@@ -4045,3 +4045,38 @@ heavy to commit. Resolved by committing the notebook *source* + the static PNG,
 the interactive path as `marimo run tools/viz/esper_viz.py` (+ the `html-wasm` export command
 for a hosted "live wing" as a deploy-time step, not committed bytes). S3 is communication
 infrastructure, not a falsifiable experiment — no gate; its bar is the 60-second-stranger test.
+
+## 2026-07-26 11:59 — S3 Increment 2: the visual surface fans out (learn-loop + rollout GIFs, MAP-Elites illumination)
+
+Increment 1 proved the Mojo→trace→renderer pipeline on one artifact (the arc_solve
+held-out triptych). Increment 2 reuses that exact pipeline for the three artifacts named
+in the plan — turning the engine's invisible behaviours into things a stranger can *see*:
+
+- **"Watch it learn" (flip_h learn-loop).** New `--learn-trace PATH` on `src/main.mojo`
+  dumps the operator's prediction on the **held-out** input every K=100 ES iters. The
+  capture is a viz-only twin, `forward_with_learning_traced` in `esper_evolution.mojo`:
+  it replicates `fit_operator`'s annealed schedule EXACTLY (same alpha/sigma rates, same
+  `evolve_fast_weights` calls) but snapshots into a **pre-allocated** buffer BETWEEN ES
+  steps — never inside the SIMD hot loop — and serializes once after the fit. The
+  production `forward_with_learning` / `fit_operator` are untouched, so no measured number
+  moves. The trace shows a clean trajectory: identity (0.00) → 0.62 by iter 500 → 1.00 by
+  iter ~3900.
+- **A repertoire behaviour rolled out (sandbox GIF).** New `src/sandbox_trace.mojo` loads a
+  MAP-Elites `.rep`, picks the highest-occupancy elite, and rolls its policy out via a new
+  viz-only `sandbox_rollout_trace` twin in `sandbox.mojo` that snapshots the whole grid each
+  tick (avatar stamped) into a caller buffer — the ES-critical rollouts stay untouched. The
+  render reads unmistakably: the avatar paints, gravity pulls the paint down, and the world
+  fills floor-up over 64 ticks.
+- **MAP-Elites illumination (zero Mojo change).** `tools/viz/render_repertoire.py` reads the
+  *existing* `.rep` binary (`EliteMap.save`) directly — avatar-final-position coverage
+  (228/256 behaviours reached) + the 4×4 mean paint field showing the gravity-well bias.
+
+Tooling decision — **GIFs with no new dependency.** No Pillow/imageio in the base venv, and
+adding one would tax the "reproduce in one command" claim. Since Esper's grids already ARE
+palette indices, I wrote a self-contained GIF89a/LZW encoder in `tools/viz/gifpng.py`
+(shared with the PNG writer, mirroring `palette.py`'s single-source ethos) — validated by a
+full encode→decode round-trip over every frame (real GIF decoders use exactly the
+one-code-early width boundary the round-trip confirmed). Everything stays numpy + stdlib.
+
+`./esper fast` stays green (3/3 held-out still solved) — the three Mojo additions are
+strictly additive/flag-gated and perturb nothing on the measured path.

docs/assets/learn.gif

@@ -1,6 +1,6 @@
 from std.memory import alloc, memset_zero, memcpy, UnsafePointer
 from std.sys import simd_width_of, size_of, has_accelerator
-from std.math import fma, exp, log
+from std.math import fma, exp, log, round
 from std.random import randn_float64
 from std.collections import List
 from std.algorithm import parallelize
@@ -410,6 +410,120 @@ def forward_with_learning(
     return result^
 
 
+# Row-major int JSON for one grid slice (rounds the Float32 cells, as the trace
+# consumers expect ints). Shared shape with arc_solve._grid_json — kept local so
+# the runtime path never imports the viz module.
+def _grid_json_ev(
+    p: UnsafePointer[Float32, MutAnyOrigin], rows: Int, cols: Int
+) -> String:
+    var s = String("[")
+    for r in range(rows):
+        if r > 0:
+            s += String(",")
+        s += String("[")
+        for c in range(cols):
+            if c > 0:
+                s += String(",")
+            s += String(Int(round(p[r * cols + c])))
+        s += String("]")
+    s += String("]")
+    return s
+
+
+# ==========================================
+# Traced forward pass ("watch it learn")
+# ==========================================
+# A viz-only twin of forward_with_learning that snapshots the evolving prediction
+# on the held-out test input every `snap_every` ES iterations, so an offline
+# renderer can show the operator converging from the identity prior to the fitted
+# transform. It replicates fit_operator's annealed schedule EXACTLY (the same
+# alpha/sigma rates and evolve_fast_weights calls) so the fit it traces is a
+# faithful ES run — the schedule mirror below MUST track fit_operator. The
+# snapshot buffer is allocated ONCE up front; each snapshot is an OperatorMemory
+# apply BETWEEN ES steps, never inside evolve_fast_weights' hot loop; the JSON is
+# assembled and returned to the caller, written to disk once after the fit. The
+# production forward_with_learning / fit_operator are untouched, so this path
+# cannot perturb any measured number.
+def forward_with_learning_traced(
+    node: UnsafePointer[HopeNode, MutAnyOrigin],
+    demonstrations: List[ArcTaskPair],
+    test_input: ArcGrid,
+    truth: ArcGrid,
+    snap_every: Int,
+    mut trace_buf: String,
+) raises -> ArcGrid:
+    var capacity = test_input.size()
+    for d in range(len(demonstrations)):
+        var in_area = demonstrations[d].input_grid.size()
+        if in_area > capacity:
+            capacity = in_area
+        var out_area = demonstrations[d].output_grid.size()
+        if out_area > capacity:
+            capacity = out_area
+
+    var workspace = ESWorkspace[OperatorMemory](capacity)
+    var rows = test_input.rows
+    var cols = test_input.cols
+    var area = test_input.size()
+    var step = snap_every if snap_every > 0 else FIT_ITERS
+
+    # One pre-allocated snapshot buffer (no per-iteration allocation): the
+    # in-loop snapshots plus a final post-fit frame.
+    var n_snaps = FIT_ITERS // step + 2
+    var snaps = alloc[Float32](n_snaps * area)
+    var snap_iter = List[Int]()
+
+    var alpha_rate = log(FIT_ALPHA1 / FIT_ALPHA0) / Float32(FIT_ITERS)
+    var sigma_rate = log(FIT_SIGMA1 / FIT_SIGMA0) / Float32(FIT_ITERS)
+    var s = 0
+    for t in range(FIT_ITERS):
+        if t % step == 0:
+            OperatorMemory.apply(node[].fast, test_input, snaps + s * area)
+            snap_iter.append(t)
+            s += 1
+        var alpha = FIT_ALPHA0 * exp(alpha_rate * Float32(t))
+        var sigma = FIT_SIGMA0 * exp(sigma_rate * Float32(t))
+        evolve_fast_weights[OperatorMemory](
+            node[].fast,
+            workspace,
+            node[].slow,
+            demonstrations,
+            FIT_N,
+            alpha,
+            sigma,
+            FIT_REG,
+        )
+
+    var result = ArcGrid(rows, cols)
+    OperatorMemory.apply(node[].fast, test_input, result.data)
+    # Final converged frame.
+    OperatorMemory.apply(node[].fast, test_input, snaps + s * area)
+    snap_iter.append(FIT_ITERS)
+    s += 1
+
+    # Assemble one JSON object: fixed input/truth + the ordered frame stack.
+    trace_buf += String('{"rows":') + String(rows)
+    trace_buf += String(',"cols":') + String(cols)
+    trace_buf += String(',"iters":') + String(FIT_ITERS)
+    trace_buf += String(',"snap_every":') + String(step)
+    trace_buf += String(',"input":') + _grid_json_ev(
+        test_input.data, rows, cols
+    )
+    trace_buf += String(',"truth":') + _grid_json_ev(truth.data, rows, cols)
+    trace_buf += String(',"frames":[')
+    for i in range(s):
+        if i > 0:
+            trace_buf += String(",")
+        trace_buf += String('{"iter":') + String(snap_iter[i])
+        trace_buf += String(',"grid":')
+        trace_buf += _grid_json_ev(snaps + i * area, rows, cols)
+        trace_buf += String("}")
+    trace_buf += String("]}")
+
+    snaps.free()
+    return result^
+
+
 # ==========================================
 # Meta-learning the slow prior (second timescale)
 # ==========================================

docs/assets/learn.png

@@ -11,10 +11,14 @@ from hope import (
     OP_DIM,
     seed_identity_operator,
 )
-from esper_evolution import forward_with_learning
+from esper_evolution import (
+    forward_with_learning,
+    forward_with_learning_traced,
+)
 from arc_io import exact_match
 from std.collections import List
 from std.random import seed, random_float64
+from std.sys import argv
 
 
 def rand_grid(rows: Int, cols: Int) -> ArcGrid:
@@ -56,7 +60,27 @@ def main() raises:
     var test = rand_grid(rows, cols)
     var truth = flip_h_grid(test)
 
-    var result = forward_with_learning(node, demos, test)
+    # Optional `--learn-trace PATH`: dump the operator's per-K-iter prediction on
+    # the held-out input as it converges (offline "watch it learn" viz). Purely
+    # additive — the default forward pass and its numbers are unchanged.
+    var trace_path = String("")
+    var args = argv()
+    for i in range(len(args)):
+        if String(args[i]) == String("--learn-trace") and i + 1 < len(args):
+            trace_path = String(args[i + 1])
+
+    var result: ArcGrid
+    if trace_path.byte_length() > 0:
+        var trace_buf = String("")
+        result = forward_with_learning_traced(
+            node, demos, test, truth, 100, trace_buf
+        )
+        var f = open(trace_path, "w")
+        f.write(trace_buf)
+        f.close()
+        print("  wrote learn-trace:", trace_path)
+    else:
+        result = forward_with_learning(node, demos, test)
 
     print("Esper forward pass complete (learned flip_h from demonstrations).")
     print(

docs/assets/repertoire.png

@@ -465,6 +465,40 @@ def sandbox_rollout_state(
     sandbox_bc(scratch_grid, r, c, bc_out)
 
 
+# Viz-only rollout twin: identical dynamics to sandbox_rollout_state, but also
+# snapshots the whole grid after each of the SB_T ticks into `frames`
+# (SB_T * SB_CELLS, caller-owned — no per-tick allocation), so an offline
+# renderer can animate the world evolving. The avatar's cell is stamped as
+# SB_AVATAR_MARK in the copy (not in the live grid) so the viewer can see where
+# the policy is acting. Separate from the ES-critical rollouts, which are
+# untouched. Returns nothing measured — pure visualization.
+comptime SB_AVATAR_MARK = Float32(11.0)
+
+
+def sandbox_rollout_trace(
+    weights: UnsafePointer[Float32, MutAnyOrigin],
+    task: SandboxTask,
+    scratch_grid: UnsafePointer[Float32, MutAnyOrigin],
+    obs: UnsafePointer[Float32, MutAnyOrigin],
+    logits: UnsafePointer[Float32, MutAnyOrigin],
+    frames: UnsafePointer[Float32, MutAnyOrigin],
+):
+    memcpy(dest=scratch_grid, src=task.grid, count=SB_CELLS)
+    var r = task.start_r
+    var c = task.start_c
+    var brush = task.start_brush
+    for t in range(SB_T):
+        sandbox_obs(scratch_grid, r, c, brush, t, obs)
+        policy_forward(weights, obs, logits)
+        var action = policy_argmax(logits)
+        sandbox_step(
+            scratch_grid, r, c, brush, task.grav_dir, task.grav_rate, action
+        )
+        var dst = frames + t * SB_CELLS
+        memcpy(dest=dst, src=scratch_grid, count=SB_CELLS)
+        dst[r * SB_COLS + c] = SB_AVATAR_MARK
+
+
 # ==========================================
 # CellSet — the coverage metric's container
 # ==========================================