EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfer

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Xingze Gao, Chuanrui Hu, Hongda Chen, Pengfei Yao, Zhao Wang, Yi Bai, Zhengwei Wu, Yunyun Han, Xiaofeng Cong, Jie Gui, Yafeng Deng, Teng Li
• Venue: arXiv: cs.AI
• Published: 2026-07-06
• Collected: 2026-07-07
• URL: http://arxiv.org/abs/2607.05202v1
• PDF: https://arxiv.org/pdf/2607.05202v1
• DOI: UNKNOWN

Plain-Language Brief

The paper introduces EvoAgentBench, a new benchmark for evaluating agent self-evolution in long-horizon learning machine systems. The benchmark focuses on procedural knowledge transfer across four domains: web research, algorithmic reasoning, software engineering, and knowledge work. It tests the reliability of transferring curated Ability content across model families but finds that no current automatic method can sustain positive gain in all settings.

What the Document Says

• Agent self-evolution in long-horizon LLM systems involves reusable procedures for searching, debugging, and verification beyond stored information.
• Current evaluations do not isolate this form of transfer as they focus on single-episode task solving or information retention.
• EvoAgentBench is a new benchmark for agent self-evolution via Ability-guided transfer across four domains: web research, algorithmic reasoning, software engineering, and knowledge work.
• The benchmark extracts trace-grounded Abilities from agent executions, canonicalizes them into operational units, and builds domain-specific Ability Graphs linking tasks that share procedural overlap.
• Every test task in EvoAgentBench is backed by verified training-side Ability support.
• The benchmark tests two scaffolds and three backbones across a 528/267 train/test split.
• Curated Ability content transfers reliably across model families, but no current automatic method sustains positive gain in all settings.
• EvoAgentBench is publicly available at https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.

Key Concepts Mentioned

• agent self-evolution
• long-horizon LLM systems
• Abilities
• procedural reuse
• benchmark
• web research
• algorithmic reasoning
• software engineering
• knowledge work
• train/test split
• scaffolds
• backbones

Claims Made by the Document

• Current evaluations do not isolate procedural knowledge transfer.
• EvoAgentBench can reliably transfer curated Ability content across model families.
• No current automatic method sustains positive gain in all settings.

Evidence Used

• The text mentions that current evaluations focus on single-episode task solving and information retention, implying they do not isolate procedural knowledge transfer.
• The paper states that curated Ability content transfers reliably across model families.
• The text states that no current automatic method sustains positive gain in all settings.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Agent self-evolution in long-horizon LLM systems is largely procedural: useful experience is not merely stored information, but reusable procedures for searching, debugging, and verification. Yet current evaluations do not isolate this form of transfer. Agent benchmarks test single-episode task solving; memory benchmarks target information retention rather than procedural reuse. We introduce EvoAgentBench, a benchmark for agent self-evolution via Ability-guided transfer across four agentic domains: web research, algorithmic reasoning, software engineering, and knowledge work. EvoAgentBench extracts trace-grounded Abilities from agent executions, canonicalizes them into operational units, and builds domain-specific Ability Graphs linking tasks that share procedural overlap. By design, every test task is backed by verified training-side Ability support. Across a 528/267 train/test split, two scaffolds, and three backbones, curated Ability content transfers reliably across model families, but no current automatic method sustains positive gain in all settings. EvoAgentBench shifts self-evolution evaluation from aggregate accuracy comparison to fine-grained diagnosis of experience encoding, routing, and uptake. The benchmark is publicly available at https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.