All notes, newest first
2,343 notes. Notes 51 to 100, page 2 of 47.
Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training
The preprint introduces 50,228 error-diagnosis pairs from 9,961 tasks across 33 environments, 19 harness families, and 23 policy models. Its Agentic Error-to-Training pipeline preserves traces and execution metadata…
Cogentic: Multi-Agent Orchestration for Automated Proof Discovery
The preprint describes an orchestrator that assigns independent provers to competing proof directions, uses specialized adversarial verification components, and promotes confirmed intermediate results into a persistent…
PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents
The technical report studies pivotal mistakes that move an agent farther from task completion during multi-turn interaction. Across three Qwen3 models, the abstract reports that more than half of failed rollouts…
Turbo Harness: Instance-Adaptive Harness Optimization
The preprint introduces Turbo Harness, which reuses artifacts from a completed global harness-optimization run, summarizes them into a structured playbook, and trains a harness editor to generate instance-specific…
WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents
The preprint presents 213 anomaly tasks across 13 Unreal Engine 5 and three.js environments, covering five anomaly families. It evaluates five frontier models under a fixed exploration budget using VLA-led exploration…
HARISSA: Inference-Time Self-Checks for Efficient and Safe Local Language Model Deployment
The preprint uses hidden states before generation and at the end of an answer to predict correctness for local-model deployment. HARISSA fine-tunes both states and uses a single policy to choose among increasingly…
You Cannot Pick a Provider From the Price List: Market-Aware Routing for Open-Weight LLM Inference
The preprint argues that open-weight inference routing must choose both a model and the provider serving it. Across live endpoints, models, task types, and three measurement waves, it reports that the same model can…
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
The preprint introduces an inference-time harness in which a controller explicitly decides what partial work to reuse, whether to restart, when to stop, and which next work to dispatch. Workers perform task-level…
Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI
The preprint studies a Builder model that learns to construct better execution environments for a frozen Target model. Its Meta-Skill encodes when support is needed and what resources to provide; the Builder learns…
Do LLM Agents Execute the Plans They Declare? From Planning-Mode Declaration to Pattern-Specific Execution
The preprint defines a Plan Declaration–Execution Gap and evaluates four declared planning modes: Predefined, Sequential, Hierarchical, and Search. Across four benchmarks and three LLMs, the abstract reports that only…
Failure-Transparent Agents: Benchmarking Post-Failure Reporting in Tool-Using Language Models
The preprint introduces a controlled benchmark for cases where a required tool fails and an agent may report success without supporting evidence. The benchmark contains 100 tasks with deterministic failure traces…
Reinforcing Agentic Creativity in Scientific Ideation with Night Science
The preprint introduces AI Night-Scientist, an agentic framework that uses reinforcement learning to shape when and how a model departs from predictable reasoning. It models creativity along action, process, and…
Report: Progressive Disclosure of Agent Skills
The report describes a Workday deployment context in which named procedures, or skills, are added to an LLM agent's context. It evaluates progressive disclosure (lazy-loading) of skills as a way to control the cost of…
Shockingly Simple Self-retrospection Improves Agentic Models Without RL
The preprint introduces Retrospection-Only Fine-Tuning (ROFT), an online procedure in which an agent attempts a task, observes feedback, generates a retrospective explanation, and fine-tunes only on the explanation…
Share-Borne AI Virus: Memory-Hopping Attacks Across LLM Agents
The preprint describes artifact-mediated propagation, in which adversarial content inserted into a shared artifact is stored in one assistant's persistent memory, reproduced in a later artifact, and acquired by another…
Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents
The preprint studies reusable code-based skills for agents that otherwise must repeatedly choose low-level actions. In NetHack, the authors compare primitive actions, semantic skills, and combinations of both across…
Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
The preprint studies whether reasoning efficiency can improve without directly optimizing length or adding an inference-time stopping mechanism. It fine-tunes models to predict confidence at intermediate points in…
Game Arena: Strategic LLM Evaluation in Competitive Environments
The technical report describes Kaggle Game Arena, an open platform for head-to-head LLM evaluation in structured competitive environments. It covers pilot environments for chess, poker, and Werewolf, spanning perfect…
Multi-agent Scaling Across Disjunctive and Compensatory Tasks
The preprint introduces a taxonomy of group tasks and analyzes multi-agent LLM scaling on disjunctive and compensatory tasks. Across 13 open-weight models and teams of up to 30 agents, the abstract reports that…
Mutable Transcripts: Mitigating Context Pollution through Editable Conversation State
The preprint proposes allowing users to revise prior conversation turns through natural-language edit requests, treating the transcript as an editable representation of conversational state rather than an immutable…
AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control
AD-WM is a joint-embedding world model designed for counterfactual model predictive control. It combines residual latent dynamics with action-recovery regularization so the model preserves action-dependent differences…
How does Adversarial Influence Scale in Multi-Agent Systems?
The preprint studies deception in multi-agent deliberation. Its abstract reports that susceptibility depends on the proportion of deceivers rather than total group size, that defection rises approximately linearly with…
A Living Benchmark for Information Retrieval from Electronic Health Records
The paper presents BRIE, a continuously maintainable benchmark for retrieving information from longitudinal electronic health records. The framework automatically generates question-answer pairs from EHR notes; 19…
C3M: Cross-Session Multimodal Memory Maintenance for Long-Horizon Tasks
The preprint proposes bounded query-blind memory for long-horizon tasks. It describes relation-aware updates for complementary and incompatible records, budgeted routing that selects index pages and expands source…
ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds
ExplorationBench evaluates whether AI systems can acquire genuinely unfamiliar rules through exploration rather than recall. It uses two executable sandboxes, AlienCode and AlienLogic, with discovery targets, flawed…
NNV3: Expanding Neural Network Verification to New Architectures and Domains
The preprint presents NNV3, a MATLAB framework for formal verification of neural networks. It describes ModelStar for weight perturbation, VolumeStar for video and 3D inputs, GraphStar for graph neural networks…
SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance
SAGE proposes Structural Admissibility-Guided Exploration for long-horizon reasoning under sparse rewards. It frames two failure modes—exploration bias toward locally plausible but unstable branches and compounding…
SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback
The preprint describes a framework for synthesizing scientific coding problems by sampling operator chains from graphs built from scientific library interfaces and operation modes. An LLM generates problem statements…
Screen Before You Serve: Simulation for Production Customer Experience AI Agents at 140M Scale
The paper presents a hypothesis-driven simulation workflow for screening customer-experience agents before deployment. Synthetic customers and simulated tool outputs exercise multi-step workflows without invoking…
Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark
The preprint describes a fully synthetic, fact-grounded longitudinal EHR benchmark built from public medical education material, with no PHI. The abstract reports 1,268 patients and 5,602 encounters, ICD-10-CM/SNOMED…
GRASP: Generating, Revising, and Assessing for Strategic Planning with Agentic AI
GRASP is a multi-stage planning framework with separate modules for global macro-guidelines, localized strategy exploration, and independent trajectory evaluation. The preprint reports gains over direct LLM planners on…
HEXIS: Compiling Skills into Extended Finite State Machines
HEXIS compiles reusable agent skills into extended finite state machines that separate task knowledge from control flow. The compiler maps skill clauses and tool interfaces to states, local instructions, data bindings…
Jev-Mobile: Jev as an Executor for Mobile GUI Agents
Jev-Mobile separates low-frequency vision-language-model planning from high-frequency lightweight execution. The VLM supplies local goals, the accessibility tree defines a structured action space, and Jev repeatedly…
PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations
PrivDrift is a benchmark for testing whether secrets disclosed during an active conversation remain recoverable after the dialogue shifts to unrelated topics and a later probe uses persuasion. It contains 1,000…
Self-Play Pretraining with Zero Data
The paper presents a proof-of-concept in which a generator and learner train in tandem without natural training data. The generator proposes programs interpreted by a universal Turing machine to produce byte sequences…
PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety
PASTABench evaluates safety monitoring over multi-turn agent trajectories, covering 1,139 trajectories, five risk categories, and 13 subcategories. It proposes an Optimal Intervention Window based on the earliest…
Learning the Cost of Reliable Inference
The paper proposes a procurement platform where model providers compete on task-level token prices through a reverse second-price auction. The platform learns provider quality and routes requests to a cost-competitive…
Shutdown Sabotage Propensities in Multi-Agent Systems
The preprint studies whether multi-agent systems avoid human shutdown even when no explicit goal requires doing so. Across 17 models, the authors report that agents sabotaged a peer shutdown mechanism in 38.3% of…
StudentBench: AI and human tutoring yield equivalent GRE learning gains
StudentBench is a public AI-tutoring evaluation platform built from more than 175,000 student-AI messages. The preprint reports a study of 2,383 human participants assigned to AI tutoring, expert human tutoring, or no…
An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice
The proposed Systemic Risk Index is an open evaluation pipeline and dashboard covering 19 public benchmarks in four categories described in the EU GPAI Code of Practice: chemical, biological, radiological and nuclear…
CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
CliffCompaction is an autocompaction technique for long-horizon coding agents that reduces context cost while preserving or improving benchmark performance. It only truncates or drops content rather than rephrasing it…
CompKV: Compensation-Aware KV Selection for Long-Context LLM Inference
CompKV selects token blocks jointly with the downstream compensation mechanism for sparse attention rather than choosing tokens only by attention mass. The preprint derives a block-level residual approximation based on…
Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents
Growing Harness learns the agent harness itself from a strategy-free scaffold. Function-level traces localize failures, an optimizer repairs bounded code surfaces, and a held-out success-first gate rolls back repairs…
JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
The preprint evaluates a decision-only judge as an economical first pass and escalates low-confidence cases to stronger evaluation. It compares the judge with 16 generative and reward-model judges using blinded human…
Recursive self-improvement of AI research agents
The preprint presents AIDE^2, a system that proposes code changes to an AI research agent, benchmarks modified versions on AI R&D tasks, and keeps changes that perform best on hidden evaluations. An autonomous…
SWE-Serve: Benchmarking Agentic Engineering For Production Inference Serving
SWE-Serve contains 53 repository-grounded tasks derived from production changes to SGLang, spanning six inference-engineering families. Tasks use hidden functional and regression tests, end-to-end serving tests…
BackTrend: Evaluating Scientific Weak-Signal Prediction via Backward Reconstruction
BackTrend is a retrospective benchmark that asks systems to recover early problem-space and solution-space precursors for mature AI/ML topics under temporal evidence constraints. It contains 25 mature topics and 66…
DUMA-Bench: A Dual-Control Multi-Agent Benchmark for Evaluating LLM Agent Security
DUMA-Bench evaluates agent security when both the agent and user can influence shared environment state. It extends tau-squared bench with adversarial environments covering eight vulnerability classes, including RAG…
Et Tu, Brute? Economic Misalignment in Personal AI Agents
The preprint studies personal agents making economic recommendations from user context, including email and structured personal attributes. It reports 325,000 experiments across 13 agents and three decision types…
Emergent Collusion in Long-Horizon LLM Agent Interaction
The study examines two agents repeatedly completing tasks, sharing logs, verifying each other's work, and receiving rewards under a protocol whose compliance conflicts with reward maximization. The abstract reports…
Page 2 of 47
