How It Works

How It Works

ACE treats an LLM's context as an evolving playbook of small, itemized bullets. Three LLM-backed roles propose changes; a deterministic (non-LLM) merge applies them; a grow-and-refine step keeps the playbook compact.

flowchart LR
    Q([Query]) --> G[Generator]
    PB[(Context Playbook)] -. injected .-> G
    G -->|trajectory + bullet usage| R[Reflector]
    FB([Feedback: labels or execution signal]) --> R
    R -->|insights, iterative refinement| C[Curator]
    C -->|delta items| M{{Deterministic Merge - non-LLM}}
    M --> PB
    M --> GR[Grow & Refine: dedupe / prune]
    GR --> PB
    classDef role fill:#1e293b,color:#fff;
    classDef store fill:#2563eb,color:#fff;
    classDef det fill:#16a34a,color:#fff;
    class G,R,C role;
    class PB store;
    class M,GR det;

Loading

---

The loop, step by step

1. Generator solves the query using the current playbook, flagging which bullets helped or misled (by id, so edits can be localized).
2. Reflector critiques the trajectory against feedback and distills concrete, reusable insights — optionally over several refinement rounds.
3. Curator turns insights into a few delta operations (ADD / UPDATE / REMOVE).
4. Deterministic merge applies those edits to the playbook — no LLM, no rewrite, no collapse.
5. Grow-and-refine de-duplicates (semantic or lexical) and prunes consistently harmful bullets.

The model only ever proposes localized edits, so accumulated knowledge can never be silently erased by a runaway rewrite.

---

The two failure modes ACE fixes

• Brevity bias — prompt optimizers collapse toward short, generic instructions and throw away hard-won domain detail. ACE counters this with a dedicated Reflector (separating evaluation from curation) and incremental deltas that add detail rather than rewriting.
• Context collapse — letting an LLM rewrite the whole context every step compresses it into a lossy summary and craters accuracy. ACE counters this with itemized deltas + grow-and-refine — the whole context is never rewritten.

examples/02_context_collapse.py reproduces context collapse with a MonolithicRewriteAgent and shows ACE staying collapse-free.

---

The three roles

Role	Input	Output	LLM?	Key property
Generator	playbook + query	trajectory, answer, bullet usage	✅	references bullets by id → localized updates
Reflector	trajectory + feedback	diagnosis + reusable insights	✅	separates evaluation from curation; iterative refinement
Curator	insights + playbook	delta operations	✅ (det. fallback)	proposes small edits; never a full rewrite
Merge	delta + playbook	updated playbook	❌	deterministic, auditable, parallel-safe

The same model can power all three roles (the paper's fair-comparison setup), or you can mix backends: ACE(generator_llm=..., reflector_llm=..., curator_llm=...).

---

Offline vs. online adaptation

flowchart LR
    subgraph Offline["Offline — system-prompt optimization"]
        TR[(Train split)] --> EP{Multi-epoch}
        EP --> ST[ACE.step] --> EP
        EP --> PBO[(Playbook)]
    end
    subgraph Online["Online — test-time memory"]
        S[Next sample] --> PR[predict] --> LE[learn] --> S
    end
    PBO -. optional warm start .-> Online
    classDef store fill:#2563eb,color:#fff;
    class PBO store;

Loading

• Offline (ACE.adapt_offline): multiple epochs over a training split to progressively strengthen the playbook. Optionally uses ground-truth labels.
• Online (ACE.adapt_online): for each test sample, predict first, then learn from the same trajectory and feedback. Can be warm-started from an offline playbook (the paper's strongest AppWorld configuration).

---

Why incremental deltas are cheap

Because the merge is non-LLM and operations are itemized:

• multiple deltas can be merged in parallel (batched adaptation);
• adaptation cost scales with the delta, not the whole context;
• long contexts amortize well at serve time via KV-cache reuse.

The paper reports up to −86.9% adaptation latency, −75.1% rollouts (offline AppWorld vs. GEPA), and −83.6% token cost (online FiNER vs. Dynamic Cheatsheet).

See Architecture for the full diagram set (bullet lifecycle, grow-and-refine algorithm, feedback regimes, data model — 14 in total).