FIELD NOTES TOPIC
scoring issues
1 note, newest first.
How Benchmarks Mis-Score Computer-Use Agents
The paper discusses issues with the scoring of Computer-Use Agents (CUAs) using brittle scripted oracles, highlighting problems such as outdated tasks, omitted decisive visual evidence, and hidden failure causes in…
