Figure: A visual taxonomy of AI agent evolution and optimisation techniques, categorised into three major directions: single-agent optimisation, multi-agent optimisation, and domain-specific optimisation. The tree structure illustrates the development of these approaches from 2023 to 2025, including representative methods within each branch.
- EvoAgentX: [https://github.com/EvoAgentX/EvoAgentX] [paper]
- (NeurIPS'22) STaR : Bootstrapping reasoning with reasoning [Paper] [Code]
- (ICLR'23) Self-consistency improves chain of thought reasoning in language models [Paper]
- (*NeurIPS'23) Tree of thoughts: Deliberate problem solving with large language models [Paper] [Code]
- (ESEC/FSE'23) Baldur: Whole-Proof Generation and Repair with Large Language Models [Paper]
- (ICLR'24) ToRA: A tool-integrated reasoning agent for mathematical problem solving [Paper] [Code]
- (Arxiv'24) NExT: Teaching large language models to reason about code execution [Paper]
- (*ICML'24)Self-rewarding language models [Paper] [Code]
- (Arxiv'24) Absolute Zero: Reinforced self-play reasoning with zero data [Paper]
- (ACL'24) Math-shepherd: Verify and reinforce llms step-by-step without human annotations [Paper]
- (AAAI'24) Graph of thoughts: Solving elaborate problems with large language models [Paper] [Code]
- (ICLR'25) Rewarding progress: Scaling automated process verifiers for LLM reasoning [Paper]
- (EACL'23) GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models [Paper]
- (ICLR'23) TEMPERA: Test-Time Prompting via Reinforcement Learning [Paper] [Code]
- (EMNLP'23) Automatic Prompt Optimization with "Gradient Descent" and Beam Search [Paper]
- (EMNLP'24) Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs [Paper]
- (EMNLP'24) GPS: Genetic Prompt Search for Efficient Few-shot Learning [Paper] [Code]
- (ICLR'23) Large Language Models Are Human-Level Prompt Engineers [Paper] [Code]
- (ICLR'24) PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization [Paper] [Code]
- (ICLR'24) Large Language Models as Optimizers [Paper] [Code]
- (ICLR'24) Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization [Paper] [Code]
- (ICLR'24) EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers [Paper] [Code]
- (ICML'24) Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution [Paper]
- (Arxiv'24) Prompt Optimization with Human Feedback [Paper] [Code]
- (Arxiv'24) StraGo: Harnessing Strategic Guidance for Prompt Optimization [Paper]
- (Arxiv'24) TextGrad: Automatic "Differentiation" via Text [Paper]
- (Arxiv'25) Self-Supervised Prompt Optimization [Paper]
- (ICML'24) A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts [Paper]
- (ICML'24) Agent Workflow Memory [Paper]
- (AAAI'24) MemoryBank: Enhancing Large Language Models with Long-Term Memory [Paper]
- (EMNLP'24) GraphReader: Building graph-based agent to enhance long-context [Paper]
- (Arxiv'24) "My agent understands me better": Integrating Dynamic Human-like Memory Recall and Consolidation in LLM-Based Agents [Paper]
- (ICLR'25) Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations [Paper]
- (ICLR'25) Boosting knowledge intensive reasoning of llms via inference-time hybrid information [Paper] [Code]
- (ACL'25) Improving factuality with explicit working memory [Paper]
- (Arxiv'25) A-MEM: Agentic Memory for LLM Agents [Paper]
- (Arxiv'25) Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory [Paper]
- (EMNLP'23) CREATOR : Tool creation for disentangling abstract and concrete reasoning of large language model [Paper]
- (ICLR'24) ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs [Paper]
- (ICML'24) Offline Training of Language Model Agents with Functions as Learnable Weights [Paper]
- (AAAI'24) Iterative tool learning from introspection feedback by easy-to-difficult curriculum [Paper]
- (ICLR'25) Offline Training of Language Model Agents with Functions as Learnable Weights [Paper]
- (Arxiv'25) Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents [Paper]
- (Arxiv'25) ReTool: Reinforcement Learning for Strategic Tool Use in LLMs [Paper]
- (Arxiv'25) ToolRL: Reward is All Tool Learning Needs [Paper]
- (Arxiv'25) Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution [Paper]
- (ICML'25) Multi-Agent Architecture Search via Agentic Supernet [Paper][Code]
- (ICLR'25) AFlow: Automating Agentic Workflow Generation [Paper] [Code]
- (ICLR'25) WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models [Paper]
- (ICLR'25) Flow: Modularized Agentic Workflow Automation [Paper]
- (ICLR'25) Automated Design of Agentic Systems [Paper] [Code]
- (Arxiv'25) FlowReasoner: Reinforcing Query-Level Meta-Agents [Paper]
- (Arxiv'25) AgentNet: Decentralized Evolutionary Coordination for LLM-Based Multi-Agent Systems [Paper]
- (Arxiv'25) MAS-GPT: Training LLMs to Build LLM-Based Multi-Agent Systems [Paper]
- (Arxiv'25) FlowAgent: Achieving Compliance and Flexibility for Workflow Agents [Paper]
- (Arxiv'25) ScoreFlow: Mastering LLM Agent Workflows via Score-Based Preference Optimization [Paper] [Code]
- (Arxiv'25) Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies [Paper]
- (Arxiv'25) MAS-ZERO: Designing Multi-Agent Systems with Zero Supervision [Paper]
- (Arxiv'25) MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programming [Paper]
- (ICML'24) GPTSwarm: Language Agents as Optimizable Graphs [Paper] [Code]
- (ICLR'24) DSPy: Compiling Declarative Language Model Calls into State-of-the-Art Pipelines [Paper] [Code]
- (ICLR'24) AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors [Paper] [Code]
- (ICLR'24) MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework [Paper] [Code]
- (COLM'24) A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration [Paper]
- (COLM'24) AutoGen: Enabling next-Gen LLM Applications via Multi-Agent Conversations [Paper] [Code]
- (Arxiv'24) G-Designer: Architecting Multi-Agent Communication Topologies via Graph Neural Networks [Paper]
- (Arxiv'24) AutoFlow: Automated Workflow Generation for Large Language Model Agents [Paper] [Code]
- (Arxiv'24) Symbolic Learning Enables Self-Evolving Agents [Paper] [Code]
- (Arxiv'24) Adaptive In-Conversation Team Building for Language Model Agents [Paper]
- (arXiv'24) Towards Better Human-Agent Alignment: Assessing Task Utility in LLM-Powered Applications — Arabzadeh et al. [Paper ]
- (arXiv'24) LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods — Li et al. [Paper]
- (2025) Enhancing LLM-as-a-Judge via Multi-Agent Collaboration (CollabEval) — Qian et al. [Paper]
- (arXiv'24) LiveIdeaBench: Evaluating LLMs’ Divergent Thinking for Scientific Idea Generation with Minimal Context — Ruan et al. [Paper ]
- (arXiv'24) Auto-Arena: Automating LLM Evaluations with Agent Peer Debate and Committee Voting — Zhao et al. [Paper ]
- (arXiv'25) MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluation — Wang et al. [Paper]
- (arXiv'24) Agent-as-a-Judge: Evaluate Agents with Agents — Zhuge et al. [Paper ]
- (arXiv'24) AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents — Andriushchenko et al. [Paper ]
- (NeurIPS'24 – Datasets & Benchmarks) RedCode: Risky Code Execution and Generation — Guo et al. [Paper ]
- (arXiv'24) MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control — Lee et al. [Paper] [mobilesafetybench.github.io]
- (arXiv'23) Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark — Pan et al. [Paper ]
- (arXiv'24) R-Judge: Benchmarking Safety Risk Awareness for LLM Judges — Yuan et al. [Paper] [rjudgebench.github.io]
- (Findings of ACL'25) SafeLawBench: Towards Safe Alignment of Large Language Models — Cao et al. [Paper ]
If you find this survey useful in your research and applications, please cite using this BibTeX:
@misc{fang2025comprehensivesurveyselfevolvingai,
title={A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems},
author={Jinyuan Fang and Yanwen Peng and Xi Zhang and Yingxu Wang and Xinhao Yi and Guibin Zhang and Yi Xu and Bin Wu and Siwei Liu and Zihao Li and Zhaochun Ren and Nikos Aletras and Xi Wang and Han Zhou and Zaiqiao Meng},
year={2025},
eprint={2508.07407},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2508.07407},
}
We would like to thank Shuyu Guo for his valuable contributions to the early-stage exploration and literature review on agent optimisation.
If you have any questions or suggestions, please feel free to contact us via:
Email: j.fang.2@research.gla.ac.uk and Zaiqiao.Meng@glasgow.ac.uk