FIELD NOTES TOPIC
reliability framework
2 notes, newest first.
How Benchmarks Mis-Score Computer-Use Agents
A research paper discusses issues with the scoring system of computer-use agents (software that navigates the web and operates desktop software). The authors argue that current scores are often inaccurate due to…
How Benchmarks Mis-Score Computer-Use Agents
The paper discusses issues with the scoring of Computer-Use Agents (CUAs) using brittle scripted oracles, highlighting problems such as outdated tasks, omitted decisive visual evidence, and hidden failure causes in…
