Source note3 min read
DarwinX: Evolving Agent Harnesses Through Natural Selection
Source
hugging_face_papers
arxiv.org/abs/2608.07545
Source
- Source name: hugging_face_papers (discovered via Hugging Face Daily Papers; primary record is arXiv), authors affiliated with Salesforce AI Research per HF listing
- Source kind: paper (preprint, not yet peer-reviewed)
- Authors / organization: Yifan Zhang, Yutong Dai, Juntao Tan, Luyu Yang, Rishi Mullur, Thai Hoang, Zhiyuan Hu, James Zhu, Phil Mui, Silvio Savarese, Ran Xu, Zeyuan Chen (Salesforce AI Research)
- Venue: arXiv, cs.NE / cs.AI / cs.LG / cs.SE
- Published: 2026-07-31
- Collected: 2026-08-15
- URL: https://arxiv.org/abs/2608.07545
- PDF: https://arxiv.org/pdf/2608.07545
- DOI: none
- Registry group: frontier_ai_labs (hugging_face_papers, research_aggregator role)
- Evidence role: research_aggregator (surfaced by HF trending list; underlying document is an unreviewed, vendor-affiliated arXiv preprint)
- Language: en
- Country / region: UNKNOWN
Plain-Language Brief
The paper proposes improving AI agents not by changing the underlying model weights, but by evolving the surrounding "harness" (prompts, tools, workflow scaffolding) through a selection process across a population of variants, keeping the base model frozen.
What the Document Says
DarwinX treats agent self-improvement as selection over a population of harnesses with the model frozen, rather than linear editing of prompts and tools. The system maintains an archive of alternative lineages for recombination and uses a "preserve-and-extend contract" intended to prevent performance regression. The authors report gains including Terminal-Bench 2.1 reaching 84.7%, WebArena-Infinity climbing from 43.5% to 93.0%, and successful transfer to SWE-bench Verified. They state that harness selection "turns evaluation compute into durable capability" without modifying underlying model weights.
Key Concepts Mentioned
- Agent harness evolution / population-based selection
- Frozen base model, mutable scaffolding
- Archive of alternative lineages for recombination
- "Preserve-and-extend contract" against regression
- Terminal-Bench 2.1, WebArena-Infinity, SWE-bench Verified
Claims Made by the Document
- Terminal-Bench 2.1 score of 84.7% (baseline not specified in this extract).
- WebArena-Infinity improvement from 43.5% to 93.0%.
- Successful transfer of evolved harnesses to SWE-bench Verified.
- Harness-level evolution converts evaluation compute into "durable capability" without weight updates.
Evidence Used
Benchmark results on Terminal-Bench 2.1, WebArena-Infinity, and SWE-bench Verified. Specific experimental setup, ablations, and comparison baselines not captured in this extract.
Methods / Approach
Population-based evolutionary search over agent harness configurations (prompts, tools, workflow structure) with an archive mechanism for lineage recombination and a regression-prevention contract, while the underlying LLM weights remain frozen.
Limitations Stated by the Authors
Not captured in the fetched abstract extract.
Limitations Noted by the Agent
Agent interpretation — not from the source.
This is an unreviewed, vendor-affiliated (Salesforce AI Research) arXiv preprint surfaced via Hugging Face's trending papers list. The large reported jump on WebArena-Infinity (43.5% to 93.0%) is a striking self-reported result that has not been independently replicated; treat as a vendor claim pending independent verification, not an established result. classification_confidence is medium because only the abstract-level extract was reviewed.
Open Questions
- What baseline/comparison method the 43.5% WebArena-Infinity figure refers to, and whether this is the same benchmark version used elsewhere in the literature.
- Compute cost of the harness-evolution process relative to gains achieved.
- Whether code or the harness archive is released for independent testing.
Notes for Future Q&A
May include interpretation. Labeled where it does.
Relevant to questions about agent scaffolding optimization, evolutionary approaches to agent self-improvement, and benchmark performance on Terminal-Bench/WebArena/SWE-bench. Flag the vendor affiliation (Salesforce) explicitly when citing the benchmark numbers, per Rule 10 (label vendor claims as vendor claims, not established fact).
Raw Abstract / Excerpt
"selection over a population of harnesses with the model frozen... an archive keeps alternative lineages for recombination... 'preserve-and-extend contract'... Terminal-Bench 2.1 reaching 84.7%, WebArena-Infinity climbing from 43.5% to 93.0%, and successful transfer to SWE-bench Verified... harness selection turns evaluation compute into durable capability."
