FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Bo Deng, Kang Zhou, Lifan Guo, Chongyang Tao, Xuanren Chen, Chenggang Xie, Renzhao Liang, Feng Chen, Chi Zhang
• Venue: arXiv: cs.AI
• Published: 2026-08-06
• Collected: 2026-08-07
• URL: http://arxiv.org/abs/2608.06144v1
• PDF: https://arxiv.org/pdf/2608.06144v1
• DOI: UNKNOWN

Plain-Language Brief

The study introduces FinEvo-Bench, a benchmark for evaluating the performance of self-evolving agents in various financial tasks. The benchmark consists of 120 real-case-grounded tasks across six financial domains, with each task being part of one out of 20 business scenes. Four different self-evolving agent scaffolds were compared using FinEvo-Bench, and it was found that the Letta scaffold achieved the highest evolved score and fewest compliance issues, while Codex showed the largest self-evolution gain.

What the Document Says

• FinEvo-Bench is a longitudinal benchmark for evaluating self-evolving agents in professional financial workflows.
• It consists of 120 tasks across 6 financial domains and 20 business scenes, each with 6 related but distinct cases.
• Each scene has a professional procedure, task facts, a rubric for task quality, and compliance evaluation.
• Four self-evolving agent scaffolds using the same Qwen3.7-Max backbone were compared.
• The Letta scaffold achieved the highest evolved score (91.65) and fewest compliance issues (0.09 per task).
• Codex showed the largest self-evolution gain (+19.37).
• Across all four scaffolds, the evolving condition improved scores by 9.33-19.37 points and reduced compliance issues by 0.12-0.44 per task.
• In Claude Code, skill-only evolution produced higher quality tasks and fewer compliance issues than memory-only and combined evolution.
• Rubric feedback led to higher scores and fewer compliance issues than reference-answer feedback.

Key Concepts Mentioned

• FinEvo-Bench
• self-evolving agents
• financial domains
• business scenes
• letta scaffold
• codex

Claims Made by the Document

• Letta achieves the highest evolved score and fewest compliance issues.
• Codex shows the largest self-evolution gain.

Evidence Used

• FinEvo-Bench was used to compare four self-evolving agent scaffolds, and Letta and Codex were found to perform well.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Most agent benchmarks evaluate tasks independently and cannot measure whether experience from one task helps with later tasks. Existing self-evolution benchmarks do not jointly cover professional workflows, open-ended deliverables, and multi-aspect evaluation. We introduce FinEvo-Bench, a longitudinal benchmark with 120 real-case-grounded tasks, 20 business scenes across six financial domains. Institution-provided professional procedures define the required operations and constraints. Eligible institution-provided and publicly documented cases supply the task facts. Each scene contains six related but substantively distinct cases that share a professional procedure and a manually reviewed rubric for task quality and financial compliance. We compare four self-evolving agent scaffolds using the same Qwen3.7-Max backbone and three independently shuffled, globally interleaved task streams. Paired non-evolving controls estimate each scaffold's self-evolution gain from retained experience, while an independent Claude Code scoring agent backed by Claude Opus 4.6 evaluates all outputs. Letta achieves the highest evolved score (91.65) and fewest compliance issues (0.09 per task); Codex achieves the largest self-evolution gain (+19.37). Across scaffolds, the evolving condition raises scores by 9.33-19.37 points and reduces compliance issues by 0.12-0.44 per task. Paired score gains at within-scene ranks 4-6 exceed those at ranks 1-3 by 6.10-8.70 points. In Claude Code, skill-only evolution produces higher task quality and fewer compliance issues than memory-only and combined memory-skill evolution. Across all four scaffolds, rubric feedback also yields higher scores and fewer compliance issues than reference-answer feedback. FinEvo-Bench measures both professional performance and self-evolution ability: how effectively an agent turns prior experience into later improvement.