failure modes
6 notes, newest first.
The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images
The study examines the 'thinking-with-images' paradigm used in multimodal Language Models (LLMs), which includes visual operations like crop-and-zoom. The research finds that these operations often result in minimal or…
Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs
The paper presents an empirical study on the effectiveness of inference-time scaling in locally deployed autonomous computer-use agents (CUAs) under strict hardware constraints. The research focuses on four dimensions…
Can AI agents conduct open-ended AI research? Early evidence from two case studies
In a study, two unpublished AI research papers were given to AI agents for completion. Despite the agents completing the engineering tasks without human help, they failed to make significant progress towards answering…
Gubernaut: A Deterministic Homeostatic Controller for Affect-Regulated LLM Agents, Validated Across Independent Model Families
The research presents a new model-agnostic controller, Gubernaut Cognitive Controller (GCC), designed to control the behavior of Large Language Model (LLM) agents. The GCC aims to mitigate reactive failure modes in…
Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
The text discusses the increasing trend of AI systems improving themselves through various methods such as revising outputs, adapting their own harnesses, and conducting AI research itself. The authors surveyed 1250…
EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning
Researchers have introduced a new benchmark called EHR-Complex to test the performance of clinical agents that interact with electronic health records (EHRs). The benchmark is built on a large dataset from MIMIC-IV and…
