EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfer

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Xingze Gao, Chuanrui Hu, Hongda Chen, Pengfei Yao, Zhao Wang, Yi Bai, Zhengwei Wu, Yunyun Han, Xiaofeng Cong, Jie Gui, Yafeng Deng, Teng Li
• Venue: arXiv: cs.AI
• Published: 2026-07-06
• Collected: 2026-07-07
• URL: http://arxiv.org/abs/2607.05202v1
• PDF: https://arxiv.org/pdf/2607.05202v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers introduce EvoAgentBench, a new benchmark for evaluating agent self-evolution in long-term LLM systems. The benchmark focuses on procedural reuse of experience, which includes searching, debugging, and verification procedures, rather than just information storage. It is based on four agentic domains: web research, algorithmic reasoning, software engineering, and knowledge work. EvoAgentBench extracts 'Abilities' from agent executions, converts them into operational units, and builds domain-specific Ability Graphs linking tasks with procedural overlap. The benchmark is designed to ensure that every test task has verified training-side Ability support. In testing, curated Ability content transfers reliably across model families, but no current automatic method consistently achieves positive gains in all settings.

What the Document Says

• EvoAgentBench is a benchmark for agent self-evolution via Ability-guided transfer across four domains: web research, algorithmic reasoning, software engineering, and knowledge work.
• The benchmark extracts trace-grounded Abilities from agent executions and builds domain-specific Ability Graphs linking tasks with procedural overlap.
• Every test task in EvoAgentBench has verified training-side Ability support.
• In testing, curated Ability content transfers reliably across model families, but no current automatic method sustains positive gain in all settings.
• EvoAgentBench is publicly available at https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.

Key Concepts Mentioned

• agent self-evolution
• Abilities
• procedural reuse
• long-horizon LLM systems
• EvoAgentBench
• web research
• algorithmic reasoning
• software engineering
• knowledge work
• Ability Graphs

Claims Made by the Document

• Current evaluations do not isolate agent self-evolution via Ability transfer.
• EvoAgentBench is a new benchmark for agent self-evolution in long-term LLM systems that focuses on procedural reuse.
• Curated Ability content transfers reliably across model families in EvoAgentBench.

Evidence Used

• The text states that current evaluations do not isolate agent self-evolution via Ability transfer.
• The text introduces EvoAgentBench as a benchmark for agent self-evolution via Ability transfer.
• The text mentions that curated Ability content transfers reliably across model families in EvoAgentBench.

Methods / Approach

• extracting trace-grounded Abilities from agent executions
• canonicalizing Abilities into operational units
• building domain-specific Ability Graphs

Limitations Stated by the Authors

No current automatic method sustains positive gain in all settings.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The agent's capabilities are limited to those defined by the authors of EvoAgentBench.

Open Questions

• What specific models or methods were used for the testing?
• How is the 'positive gain' measured in the evaluation?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The text does not specify the specific models or methods used for the testing.
• The term 'positive gain' refers to an improvement in the performance of the agent, but the exact metric is not specified.

Raw Abstract / Excerpt

Agent self-evolution in long-horizon LLM systems is largely procedural: useful experience is not merely stored information, but reusable procedures for searching, debugging, and verification. Yet current evaluations do not isolate this form of transfer. Agent benchmarks test single-episode task solving; memory benchmarks target information retention rather than procedural reuse. We introduce EvoAgentBench, a benchmark for agent self-evolution via Ability-guided transfer across four agentic domains: web research, algorithmic reasoning, software engineering, and knowledge work. EvoAgentBench extracts trace-grounded Abilities from agent executions, canonicalizes them into operational units, and builds domain-specific Ability Graphs linking tasks that share procedural overlap. By design, every test task is backed by verified training-side Ability support. Across a 528/267 train/test split, two scaffolds, and three backbones, curated Ability content transfers reliably across model families, but no current automatic method sustains positive gain in all settings. EvoAgentBench shifts self-evolution evaluation from aggregate accuracy comparison to fine-grained diagnosis of experience encoding, routing, and uptake. The benchmark is publicly available at https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.