Esper — a bare-metal, pure-Mojo neuro-symbolic reasoning engine. Fourteen pre-registered gates, 2026-07-19 → 2026-08-25. Three GO, one PARTIAL, ten STOP or NOT-TESTABLE.
Summary. Esper learns a transformation in-context from a handful of demonstration pairs by fitting a small memory with a derivative-free Evolution Strategy — no backprop, no autodiff framework, no symbolic DSL — and is scored only on held-out examples it never saw during the fit. This report covers a fourteen-gate campaign asking what carries from one world to the next. The headline is a negative and it is the point: flat weight priors do not transfer, a hand-designed relational retrieval index retrieves worse than the plain baseline it was meant to beat, and for most of the campaign the measuring instrument could not resolve a transfer effect at all — which turned out to have a specific, findable cause. What did survive is narrower and better evidenced than what the campaign set out to show: the learning result carries across three different data substrates through an unmodified learning core, and retrieval across a world gap works even where every attempt to improve it failed.
The methodology is the other half of the contribution. Every number here was committed to — metric, threshold, seeds, and the control that could take the claim away — in a frozen document written before the run. Five of the fourteen gates ended in seconds, for zero compute, because a cheap precondition checked first said the question could not be asked.
| The question asked | The answer measured | Gates |
|---|---|---|
| Does a flat weight prior transfer from world N to world N+1? | No. What transfers is a behavioural repertoire plus its index. | 001, 002 |
| Can a hand-designed relational key improve retrieval across that gap? | No — it retrieves worse than the plain baseline. | 010, 011 |
| Why can't the transfer instrument resolve an effect? | Independent full-budget fits land in a genuine single basin — not under-search, not a blind read-out, not a policy-class floor. | 008, 009, 012 |
| Does the in-context learning result depend on the data substrate? | It carries to three substrates, one of them topology-free, through an unchanged learning core. Not "substrate-neutral in general" — see §4.3. | 006, 007, 013, 014 |
What is not answered: the campaign's own north-star question — what carries from world N to world N+1 — is open. No gate answered it, and this report does not claim otherwise.
Three layers, following the Nested Learning / HOPE architecture:
- Slow weights — a meta-learned prior over memories, updated rarely (a Reptile outer loop).
- Fast weights — the memory itself, fit during the forward pass by an annealed, antithetic, SIMD/FMA Evolution Strategy on the task's demonstration pairs.
- An arena — a move-only bump-pointer allocator hosting POD nodes and their weight slices.
Three constraints are load-bearing for reading the results:
- No backprop and no ML framework. All learning is done by ES. This is not a performance claim; it is what makes the fast/slow split cheap enough to run per-task on commodity hardware.
- No symbolic DSL on the runtime path. Geometric, colour and count transforms must emerge as fitted parameters. Hand-written transforms exist only in the offline generator, as ground truth to rediscover. A result reached by handing the engine the answer is recorded as a negative.
- Held-out generalization is the only metric. Fit on a task's train pairs, score the unseen test pair. Uncheatable by memorization.
A gate is a frozen document written before the run that fixes: the metric and what downstream component actually consumes it; the GO / STOP / NOT-TESTABLE thresholds; the seed plan; the refuting control — one that would read ~nothing if the claim were false; and the disclosures, i.e. the ways the gate is weaker than it looks. Once registered it is never edited. If a later gate discovers the registration was wrong, the correction is recorded in the successor, and the original stands as written.
Two rules do most of the work:
- Match the metric to its consumer. A metric that looks adjacent to the one your consumer reads can be decoupled from it. A pre-gate-era milestone (T-POC-2, 2026-07-15, before this campaign) died exactly here: its learned model ordered candidates acceptably (Kendall τ ≈ 0.4–0.7, against a scrambled control at ≈ 0.1) but the component downstream never consumed an ordering — it followed the argmax, and top-1 regret was broken. One level down, the same trap: one-step prediction accuracy rose 0.0 → 0.93 while 64-tick rollout usefulness got worse.
- Name the cheat you expect to be tempted by, in advance. Gate 014 registered, before any number existed, that if its equivalence metric returned 1.0 the temptation would be to perturb the data family until it dropped so the gate "produced a result". Pre-naming a specific cheat converts a rationalisation into a visible rule violation, which is substantially more effective than a general commitment to honesty.
GO and STOP are the ordinary outcomes. PARTIAL is a split result under multiple registered objectives. NOT-TESTABLE is the one that carries the methodological weight: it means the instrument could not be asked the question — a control that cannot diverge, a precondition the data cannot meet — so no arm was fit and nothing was scored. A NOT-TESTABLE is not a soft "no". It closes the specific lever it was aimed at and is explicitly not an entitlement to redesign and try again.
Each gate puts its falsifiable preconditions in an ES-free ladder checked first. Five gates — 006, 008, 010, 012, 014 — terminated there, before any fit ran. Gate 012 returned in 7 seconds for zero ES budget. Gate 011's screening round cost 11 seconds and killed an entire family of candidate controls.
The economics are worth stating plainly: on this hardware a full fit block runs for hours, so a precondition that fires costs seconds and saves an afternoon — and, more importantly, saves you from a number you would have believed. Gate 014's ladder prevented a specific concrete error: a bug in the harness (§5) would otherwise have bought ~2 hours of compute and produced a clean-looking GO that was a bookkeeping artefact.
Gate 001 (PARTIAL) / Gate 002 (STOP). Warm-starting a new world from a prior world's weights was tested under two registered objectives and closed under both. All three of Gate 002's STOP conditions triggered across three seeds. What did show a transfer signal was the repertoire's index — retrieval — not the weight vector.
Three gates then tried to rescue the measurement and all three failed, which is how the campaign learned the problem was not where it was looking:
- Gate 003 (NOT-TESTABLE). The transferability proxy is banked but undecidable on existing
data:
N_dec = 5against a frozen floor ofN_dec ≥ 6. Its threshold (τ = 0.150423175) and scrambled-prior control are kept for reuse when more worlds growN_dec. - Gate 004 (STOP). Write-everything retention earns nothing on this world family — the two-source arm ties both single sources and the equal-count scrambled control.
- Gate 005 (STOP). Scoring the fit trajectory instead of its endpoint was pre-registered as the fix for saturation, and failed. The saturation was not a metric problem.
Across four gates the noise floor came back NF = 0.0: independently-seeded, full-budget
fits converge to the same optimum, so most goals never become decidable and no transfer
effect can be resolved. Two candidate explanations were tested and both were falsified.
Gate 008 (NOT-TESTABLE) falsified the world-difficulty explanation. Four world/budget
knobs had been frozen on the theory that the worlds were too easy. The pre-fit sensitivity
clause fired at 6 of 6 gating cells: the base family already flips terminal occupancy on
0.56–0.88 of one-action perturbations. The knobs do add a real channel — at +0.018 to +0.038
against a ≥ 0.20 bar, about 6× under. World difficulty was never the binding constraint.
Gate 009 (GO) named the mechanism. The tie event decomposes into the exact identity
TIE_end = COLLAPSE + BLIND_abs + ISO, and at 3 of 3 layout seeds, on two world kinds and all
five screened random-number bases, COLLAPSE carries 0.909 / 1.000 / 0.957 of the tie
mass — against a displacement-matched control at 0.667 / 0.792 / 0.770, i.e. 4–8× its own
bar. So the ties are a genuine single basin. They are not under-search; not read-out
blindness (BLIND_cond = 0.0 in all nine cells — the behaviour descriptor never once
collapsed two distinct terminal states); and not a policy-class floor. CONV_med ≈ 0.65
against a null of 1.0 places it precisely: the fits stay far apart in parameter space and
land on identical behaviour. The policy→behaviour map is many-to-one.
An erratum from the same gate sharpened it further: the basin is a basin in avatar pose, not in world state. The action trace differs from the zero policy on all 64 ticks while the terminal grid is median-identical, and where two fits differ at all, 100% of the squared gap sits in the 2 pose dimensions in five of nine cells.
Gate 012 (NOT-TESTABLE) spent the one lever this licensed, and lost it. The licensed
treatment — an occupancy-dominant goal-admission rule — works and still misses: it lifts the
median occupancy share 6–8× on exactly the cells that fail, but on the scatter layouts the
distribution starts an order of magnitude too low to reach dominance (0.26–0.39 against a 0.50
precondition, at 3 of 3 independent layouts). The published share distribution says why, and
it is a new measurement rather than a re-reading: the bottom 30–40% of candidate goals have
an occupancy share of exactly 0.0 — a fresh random policy rolls out 64 ticks and leaves block
occupancy bitwise equal to the zero policy's, moving only the avatar's pose. The world family
cannot supply occupancy-demanding goals at the density the treatment needs. Registered
disclosure 3 predicted this outcome before any number existed.
That gate's licensed lever is therefore spent, unmeasured — which is a loss, and is booked as one rather than as a discharge.
The question: does an in-context learning result proved on grids carry to a different data substrate through the same, unmodified learning core?
- Gate 006 (NOT-TESTABLE). The degree-blind controls were toothless — a blind ceiling of ≈ 0.99 means a control that cannot fail proves nothing.
- Gate 007 (GO). Re-aimed at a variable-degree graph substrate:
after(SUM) = 1.0 / 1.0 / 0.9993at seeds 0/1/2, with both degree-blind controls failing (0.48–0.57 and 0.50–0.62), through an unmodified ES core. Registered caveats carried: measured at effective alphabet 2 and rule length 2, a narrower world than the grid family, so grid parity is not claimable unqualified. - Gate 013 (GO). A third substrate — a run-segmented sequence that hands the memory no
topology at all, where a neighbourhood must be derived from the data's own run structure.
after(SUM) = 1.0 / 1.0 / 1.0, both degree-blind controls failing (0.371–0.563, 0.422–0.551), the across-seed non-overlap clause clearing by 0.449, at an identical model size on all three substrates. Notably, the gate's own registered prediction was falsified: it predicted NOT-TESTABLE on the grounds that coarser bucketing would push the blind ceiling past 0.85; it came back 0.802–0.812. A prediction booked before the number and refuted by it, in the direction that costs compute rather than saving it. - Gate 014 (NOT-TESTABLE) — and this is the bound. The fourth substrate was a set: adjacency removed entirely. Designing it revealed that the answer was determined before any fit could run. The memory reduces a neighbourhood with a sum, which is permutation- invariant, so it reads exactly two things — the symbol multiset and the degree — and nothing downstream is ever given order, identity or adjacency. A bag of size k presents what a run of length k presents. The gate was re-aimed at the question a measurement could still settle — is the set substrate an information-preserving re-encoding of the sequence one? — and returned 1.0 at all three seeds. The equivalence is now a checked property of the code rather than a plausible paragraph.
The sanctioned claim, and the sentence it may not be stretched into. This is three substrates, one of them topology-free — never "substrate-neutral in general". A run is a clique, so the sequence substrate is data-equivalent to a graph with clique components, and the set substrate is a re-encoding of the sequence one. No further substrate can add a data point while the memory's read is a neighbourhood multiset. The open question moved from the substrate to the read.
Gate 007's GO licensed one consequence: a relational retrieval key built over the graph substrate. Registering it immediately refused the obvious version — naming relations (blocked-by-wall, pushed-into-wall, …) is precisely the hand-installed vocabulary the no-DSL constraint forbids. The registered key was structural instead: the full class-adjacency matrix over the state's own sign classes, every entry present and not one selected.
Gate 010 (NOT-TESTABLE) fired at 9 of 9 cells — but on the null, not on the refuter. The descriptor and the structure-destroying control were both live and separating; what failed was a control that turned out to be an information-preserving re-weighting of the baseline on the same space. It was matched on dimension and scale when it needed matching on perturbation magnitude. No bar was relaxed: relaxing one after watching a control fail it is the archetypal post-hoc knob, and a correct-looking argument found after seeing the number is indistinguishable from a motivated one.
Gate 011 (STOP). The single licensed re-roll was spent on the null's design alone. An 11-second
screening round falsified the fix the previous gate's own diagnosis implied — scaling the
projection cleared 1 of 9 cells — and two replacement controls were adopted on measured results.
Then the treatment lost: WIN peaked at 0.619 against a 0.70 GO bar and sat at or under the
0.55 STOP trigger at every seed. Three things the controls add, and they are why this is a
finding rather than a failure:
- The instrument was live. Plain retrieval beat cold at 8 of 9 cells — retrieval across the world gap genuinely works. This is a working mechanism returning a "no".
- The registered weakness was confirmed at no cost. The state-independent control was the worst arm everywhere, exactly as disclosed in advance.
- The scrambled graph was not reliably worse than the true one. In several cells the scrambled structure retrieved better — the sharpest available statement that the descriptor's structure carried no retrieval-relevant information.
The mechanism is dilution: 20 arrangement coordinates outvoting 18 occupancy ones, with arms ranking by how much they perturb the answer rather than by what they know. Registered disclosure 1 had named the likely reason in advance — the goal is specified as an occupancy descriptor, so occupancy-matching matches the very quantity the task asks for, while arrangement describes a property no goal mentions.
The campaign's most transferable finding is not about transfer. In a single week, three separate instruments appeared to measure something and measured nothing. Each was caught by an independently-derived expectation of what the instrument should say.
-
A control that was the baseline in disguise (Gate 010). It was matched on dimension and scale and looked like a different key; it was a linear re-encoding of the same 18 numbers. It moved the answer on 6–11 of 24 goals where a key carrying genuinely new coordinates moves it on 20–23. Caught because the other controls cleared and this one didn't — a pattern that only reads as diagnostic if you registered what each control was for.
-
A pairing bug that looked like a discovery (Gate 014). The equivalence metric first came back 0.588 / 0.617 / 0.617. Read naively that is a headline: the substrates differ, the structural argument is wrong, the expensive fit block is required. It was a bug — element twins were paired by rank within a bag, but a neighbourhood multiset excludes the element's own symbol, so mis-pairing changes the excluded symbol about half the time at this alphabet size, which is exactly the 0.6 that came out. The registered prediction is what caught it. Disclosure 1 said the value should be 1.0 by construction; the number disagreed; the disagreement was therefore evidence about the harness, not about the world. Without a pre-registered expectation, 0.6 is just a number — and it is a plausible number, the kind that gets believed.
-
A format check blind to the files it existed to check. The pre-commit check ran the formatter and then
git diff— which does not show untracked files, i.e. is blind to precisely the files a new module consists of. It reported "clean" for a tree in which all four files of a new harness were unformatted. The replacement never consults version control at all: it hashes every source file on disk, formats, re-hashes, and reports what changed — and continuous integration now runs literally the same script rather than an equivalent one. It was verified by reproducing the original failure, not by inspection.
The common lesson: a check that passes for the wrong reason is worse than no check, because it retires the question. No check leaves you suspicious; a false pass leaves you confident. The defence in all three cases was the same, and it is cheap: derive what the instrument should say before reading what it does say.
Stated at the strength the evidence supports, no further:
- In-context learning without backprop works and generalizes held-out. A small memory fit by ES on a task's demonstration pairs reaches exact match on unseen test pairs for synthetic transforms, with a meta-learned prior as the anchor.
- The learning result is substrate-portable across three substrates, one of them
topology-free, through an unmodified learning core and an identical model size — bounded by
the clique equivalence and by the memory's
(multiset, degree)read. - Retrieval across a world gap is live; every hand-designed attempt to improve its index has failed. The index question is not closed — only these ways of attacking it are.
- Flat weight priors do not transfer. Closed under both registered objectives.
- The transfer instrument's saturation has a named cause — a genuine single basin in the policy→behaviour map — and it is not fixable by a smarter metric, and not caused by the worlds being too easy.
- A reward-free open-endedness ladder runs end to end through the same unchanged ES core: novelty search → a persistent MAP-Elites repertoire → exact empowerment → a world model with noise-immune learning progress → few-shot transfer of an unsupervised repertoire → an emergent curriculum reaching 0.154 held-out where domain randomization reaches 0.0.
- It does not show cross-world transfer. The north-star question is open. No gate has measured a transfer curve, and the two most recent verdicts on that Road license no next step.
- It does not show substrate-neutrality in general (§4.3).
- The substrate results carry narrow caveats — effective alphabet 2, rule length 2 — inherited unchanged across Gates 007, 013 and 014, and grid parity is not claimable unqualified.
- It is not competitive on the ARC-AGI-2 scoreboard: 41/1000 on train and 0/120 on the real eval set. That is reported because it is the number, not because it is flattering.
- One debt is outstanding: a reproduction anchor on Gate 012's harness sits inside a fit block that never executed, so it is undischarged, and the first successor to reach that block inherits the obligation.
Everything is one command from a bootstrapped shell; there is no cluster and no large corpus anywhere in this campaign, by design. The full suite runs in ~10 minutes on a developer machine.
./esper suite # full test suite — every milestone proof
./esper fast # ~2 min gate: a strict subset at FULL budget, no relaxed thresholdsThe gates themselves are in docs/gates/, frozen as registered, each with its result
section appended after evaluation. The chronological narrative — including every blocker and its
diagnosis — is in docs/journal/. The canonical direction is
docs/ROADMAP.md.
| Gate | Question | Verdict | Fit block ran? | Date |
|---|---|---|---|---|
| 001 | world-model prior transfer | PARTIAL | yes | 2026-07-19 |
| 002 | prior rollout | STOP | yes | 2026-07-19 |
| 003 | transferability proxy | NOT-TESTABLE | no — retrofit over existing runs | 2026-07-21 |
| 004 | case-based retention | STOP | yes | 2026-07-21 |
| 005 | anytime transfer metric | STOP | yes | 2026-07-27 |
| 006 | graph substrate port | NOT-TESTABLE | no | 2026-07-27 |
| 007 | wide-degree graph substrate | GO | yes | 2026-07-28 |
| 008 | world-difficulty calibration | NOT-TESTABLE | no | 2026-07-28 |
| 009 | fit degeneracy | GO | yes | 2026-07-30 |
| 010 | relational retrieval key | NOT-TESTABLE | no | 2026-08-04 |
| 011 | relational key vs matched nulls | STOP | yes | 2026-08-04 |
| 012 | goal set vs initialization | NOT-TESTABLE | no | 2026-08-23 |
| 013 | sequence substrate port | GO | yes | 2026-08-25 |
| 014 | set substrate port | NOT-TESTABLE | no | 2026-08-25 |
Ten of fourteen verdicts are negative. Five gates (006, 008, 010, 012, 014) ended inside their ES-free ladder before any arm was fit, and a sixth (003) was a retrofit over runs that already existed. That shape is deliberate and is kept visible: the negatives are what steered every direction change in this project.