FIELD NOTES TOPIC
failure analysis
3 notes, newest first.
How Benchmarks Mis-Score Computer-Use Agents
A research paper discusses issues with the scoring system of computer-use agents (software that navigates the web and operates desktop software). The authors argue that current scores are often inaccurate due to…
Failures Reveal What Metrics Miss: An Evidence-Driven Agent for Recursive Refinement of ECG Classifiers
Researchers present a new framework called RecursiveECG that uses a language model (LLM) to automatically design ECG classifiers by learning from specific failures and objective ECG evidence. The LLM is used as an…
Concept-Guided Spatial Regularization for World Models in Atari Pong
Researchers evaluate five world model agents in Atari Pong and find inconsistencies and failures in the generated video trajectories. They hypothesize that insufficient modeling of task-critical concepts may contribute…
