FIELD NOTES TOPIC
ai benchmarks
3 notes, newest first.
MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations
The paper introduces MemOps, a new benchmark for evaluating the lifecycle of memory operations in long-horizon conversations for LLM-based agents. Unlike existing benchmarks that focus on final answers, MemOps focuses…
ClassicLogic: A Knowledge-Driven Benchmark of Classic Puzzle Games for Evaluating Compositional Generalization
The paper introduces ClassicLogic, a new benchmark suite designed to evaluate an agent's ability to learn and compose problem-solving strategies in four classic logic puzzles: Sudoku, KenKen, Kakuro, and Futoshiki. The…
EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfer
The paper introduces EvoAgentBench, a new benchmark for evaluating agent self-evolution in long-horizon learning machine systems. The benchmark focuses on procedural knowledge transfer across four domains: web…
