Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows

Source

arXiv

arxiv.org/abs/2608.06144v1

Read the source

Source

Plain-Language Brief

Researchers have created a new benchmark for self-evolving agents in financial workflows called FinEvo-Bench. The benchmark includes 120 real-life tasks across six domains, with each task having a specific professional procedure and rubric for quality assessment. Four different self-evolving agent scaffolds were tested on the benchmark using the same backbone, and results showed that Letta achieved the highest evolved score while Codex had the largest self-evolution gain. The research also found that skill-only evolution produced better task quality and fewer compliance issues than memory-only or combined memory-skill evolution.

What the Document Says

  • Researchers created a benchmark called FinEvo-Bench for self-evolving agents in financial workflows.
  • FinEvo-Bench consists of 120 real-life tasks across six domains and includes professional procedures, rubrics, and factual information from institutions or publicly documented cases.
  • The benchmark has 20 business scenes, with each scene containing six related but distinct cases that share a professional procedure.
  • Four self-evolving agent scaffolds were tested using the same Qwen3.7-Max backbone.
  • Letta achieved the highest evolved score and Codex had the largest self-evolution gain.
  • Codex had an increased self-evolution gain of +19.37 compared to non-evolving controls.
  • Across scaffolds, the evolving condition improved scores by 9.33-19.37 points and reduced compliance issues by 0.12-0.44 per task.
  • In Claude Code, skill-only evolution produced higher task quality and fewer compliance issues than memory-only or combined memory-skill evolution.
  • Rubric feedback resulted in higher scores and fewer compliance issues compared to reference-answer feedback.

Key Concepts Mentioned

  • self-evolving agents
  • Financial workflows
  • FinEvo-Bench
  • benchmark
  • 120 real-life tasks
  • six domains
  • 20 business scenes
  • Letta
  • Codex
  • Qwen3.7-Max
  • Claude Code
  • skill-only evolution
  • memory-only evolution
  • combined memory-skill evolution
  • rubric feedback
  • reference-answer feedback

Claims Made by the Document

  • FinEvo-Bench is a useful benchmark for self-evolving agents in financial workflows.
  • Letta has the highest evolved score among the tested scaffolds.
  • Codex has the largest self-evolution gain among the tested scaffolds.

Evidence Used

  • The researchers created and evaluated FinEvo-Bench for this purpose.
  • Results from the experiments show that Letta achieved the highest evolved score, and Codex had the largest self-evolution gain.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Most agent benchmarks evaluate tasks independently and cannot measure whether experience from one task helps with later tasks. Existing self-evolution benchmarks do not jointly cover professional workflows, open-ended deliverables, and multi-aspect evaluation. We introduce FinEvo-Bench, a longitudinal benchmark with 120 real-case-grounded tasks, 20 business scenes across six financial domains. Institution-provided professional procedures define the required operations and constraints. Eligible institution-provided and publicly documented cases supply the task facts. Each scene contains six related but substantively distinct cases that share a professional procedure and a manually reviewed rubric for task quality and financial compliance. We compare four self-evolving agent scaffolds using the same Qwen3.7-Max backbone and three independently shuffled, globally interleaved task streams. Paired non-evolving controls estimate each scaffold's self-evolution gain from retained experience, while an independent Claude Code scoring agent backed by Claude Opus 4.6 evaluates all outputs. Letta achieves the highest evolved score (91.65) and fewest compliance issues (0.09 per task); Codex achieves the largest self-evolution gain (+19.37). Across scaffolds, the evolving condition raises scores by 9.33-19.37 points and reduces compliance issues by 0.12-0.44 per task. Paired score gains at within-scene ranks 4-6 exceed those at ranks 1-3 by 6.10-8.70 points. In Claude Code, skill-only evolution produces higher task quality and fewer compliance issues than memory-only and combined memory-skill evolution. Across all four scaffolds, rubric feedback also yields higher scores and fewer compliance issues than reference-answer feedback. FinEvo-Bench measures both professional performance and self-evolution ability: how effectively an agent turns prior experience into later improvement.