All notes, newest first
2,343 notes. Notes 1 to 50, page 1 of 47.
Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs
Across 18 base-instruct model pairs spanning Qwen, Llama, and Gemma, the authors report that harmfulness is more readily decoded from attention updates while MLP updates contribute more strongly to harm-selective…
DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design
DRYLAB-Bench evaluates agentic mutation-design capability in 30 hazardous scenarios distilled from biosecurity guidelines, using multi-round tool-augmented interactions. It scores designs against 187,695 empirical…
A Privileged Teacher That Only Scores Cannot Search: Critique-Conditioned On-Policy Self-Distillation for Reasoning
The study tests whether scoring signals in on-policy self-distillation localize errors and push probability away from erroneous student tokens. It reports that full-distribution KL and token log-ratio signals do…
RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
RATED combines a frozen dual encoder for semantic matching with a lightweight contextual-bandit module that reranks tools using real-time execution feedback. The authors introduce EcoAgent, a benchmark of 5,172 tools…
X-AgentRed: Red-Teaming Personal Agents in Evolving Multi-Service Ecosystems
X-AgentRed constructs a directed graph of tool dependencies, effects, harms, and execution preconditions to discover harmful cross-service action paths. Execution feedback expands and refines the paths through tool…
BrickBench: Evaluating Agentic Brick Design
The preprint introduces BrickBench, a benchmark for text-conditioned LEGO-set design. Agents must produce assemblies that satisfy semantic and design criteria while remaining physically buildable, selecting parts from…
Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
The preprint evaluates epistemic humility through three trajectory-level behaviors: Identify, Solve, and Escalate. It tests controlled and naturally occurring conflicts between parametric knowledge and retrieved…
Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents
The preprint proposes Hippocam, a hierarchical memory and continual-learning architecture that organizes ongoing work as nested intents. Active context stays centered on the current intent, while completed intents are…
OnTrack: Real-Time Monitoring and Intervention in LLM Agent Trajectories via Streaming Structure-Aware Optimal Transport
The preprint proposes OnTrack, a streaming monitoring mechanism that compares an agent's steps and dependencies against recorded successful runs to alert users or block the agent. It evaluates full-reference…
When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation
The preprint proposes Reasoning Adaptation through Cross-Turn Estimation (RACE), which estimates when prior reasoning remains sufficient for subsequent actions. Its LoGiC procedure identifies reasoning turns whose…
Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoff
The preprint develops a population-level model in which agent fitness depends on cybersecurity capability. It reports that without collaboration, takeoff requires individual capability above a critical threshold; with…
GeoReform: Reflective Formalization Evolution for Multimodal Geometry Problem Solving
The preprint proposes GeoReform, which treats geometric formalization as an optimizable policy rather than a fixed parser output. It collects failed rollouts, diagnoses representation defects, and mutates how entities…
Searching for "Harmful Refusal": A Psychometric Audit of an AI Safety Benchmark
The preprint audits whether harmful refusal is a single measurable attribute in HELM Safety. It reports saturation in three candidate datasets and uses multidimensional item-response theory plus differential item…
Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness
The preprint tests whether legal models' named statutes or precedents causally support their verdicts. Holding case facts fixed, it substitutes the named authority and examines verdict changes and hidden-state dynamics…
On the estimation and validity of AI time horizons---a statistical look at the METR plot
The preprint recomputes METR-style 50% AI time horizons over 228 tasks and 26 AIs using splines and item-response theory. The authors report that the fitted spline is nearly flat from roughly 2–30 minutes and close to…
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution
EvoAlloc is described as a self-evolving resource-allocation agent for LLM-based program evolution. It consolidates prior search and allocation outcomes into reusable experience, revises its allocation strategy, and…
HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling
The preprint frames multimodal synergy as higher-order dependence that can be lost by pairwise fusion. HRIL uses an empirical cross-moment tensor with Tucker decomposition and a synergy-aware regularizer. The authors…
Recursive Self-Improvement through Multi-Agent Self-Supervision
The work describes MASS, which alternates evolutionary workflow optimization with supervised fine-tuning on self-generated trajectories. The abstract reports discovery of multi-agent workflows and, over two cycles with…
Prior or Feedback? What an LLM Uses When Adapting Neural Operators
The study tests whether an LLM scientific agent adapts neural-operator tuning using experimental feedback under a limited trial budget. The authors report lower held-out nRMSE than random search and Bayesian…
Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems
The preprint tests regulatory-rule sensitivity by deleting, swapping, or negating a rule while holding the case fixed across five models and 20 domains. It reports that verdicts are often invariant to large rule…
nanoMuse: An Open-Source Personal Agent for Every Device You Own
The preprint presents nanoMuse as an open-source counterpart to persistent personal agents that operate across a user's accounts and devices. It describes a shared conversation over a user-runnable relay, a Sentinel…
ScienceClaw: Benchmarking Continual Self-Evolution of AI-for-Science Agents Across the Natural and Social Sciences
The preprint formalizes fixed-parameter program self-evolution that links task solving, scientific verification, and program updates. ScienceClaw-Eval spans 23 disciplines and measures correctness, evolutionary gain…
Sherpa: Teaching LLMs to Teach Adaptively
The preprint introduces Sherpa, a multi-turn reinforcement-learning framework that trains an LLM teacher against multiple student archetypes with distinct learning preferences. The teacher is optimized directly against…
Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations
The preprint presents Transect, an open-source package built on Inspect Scout for analyzing long agent runs whose transcripts can span hundreds of pages. Configurable evaluation families align recorded events, token…
VeriFine: Scaling Verification for Self-Improvement in Embodied Reasoning
The preprint proposes an agent harness that co-evolves the policy, training curriculum, and judge. A Policy Improvement Loop diagnoses recurring failures and builds an adaptive curriculum; when verification becomes a…
Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?
The preprint introduces BOTTLED, a benchmark in which agents receive an unlabelled workload and must complete it under fixed time, compute, and LLM API budgets. Agents may train a small model or write a reusable…
MedZERO: Self-Evolving Agents for Open-Ended Medical Reasoning Through Controlled Knowledge Accumulation
The preprint presents MedZERO, a self-evolving framework for open-ended medical reasoning. An Examiner generates medical question-option pairs, while a Reasoner solves them with evidence-grounded multi-turn reasoning…
ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding
The preprint introduces a benchmark for unified evaluation of risk-treatment capabilities in coding agents. It maps Avoidance, Transfer, Mitigation, and Acceptance from software risk management to 200…
SquidAgent: Parallelize Wisely, Coordinate Efficiently
The preprint identifies re-exploration and alignment as hidden costs that can make parallel multi-agent systems slower than a single-agent baseline. It proposes a token-based decision criterion, direct forking from the…
WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?
The preprint introduces WorldSolver, a benchmark of 168 simulation tasks derived from physical phenomena in 61 classic computer-graphics papers across seven physical domains. Each task provides a code scaffold while…
AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy
The preprint introduces AgentPrivArena, a reproducible environment that integrates authentic MCP tools and self-hosted services to evaluate privacy risks during realistic multi-step agent execution. It proposes…
What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents
The preprint introduces AgentMonBench, a software-engineering benchmark for judging requirement/behavior alignment and identifying consequential autonomous decisions that warrant verification. Its Evidence-Grounded…
Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM Serving
The preprint proposes HEAR, a bidirectional Harness–Engine Pairing protocol that lets an agent harness communicate workflow intent and execution requirements while the inference engine returns runtime state…
HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention
The preprint introduces HERA, a framework that constructs verifiable feasible/infeasible task pairs through controlled environment mutations and adapts the agent harness toward prior weaknesses. The abstract reports…
ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement
The preprint describes an autonomous post-training harness that attributes errors, selects update directions, and executes model updates under a limited compute budget. It combines metric- and case-level analysis…
Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents
The preprint proposes Dependency-Aware Group Policy Optimization (DepGPO) for multi-step terminal tasks. It builds a command dependency graph from execution traces, traces backward from resources inspected by a…
From Benchmarks to Production: A Text-to-SQL System for Complex Financial Data
The preprint describes Financial LINking Text-to-SQL (FLINT), a domain-specialized system for financial databases whose concepts are stored as opaque integer keys. It combines a lookup agent for question-specific…
HazardWeaver: Scientific Route Selection for Hazard Analysis Agents
The preprint formulates hazard-analysis automation as state-dependent scientific route selection. HazardWeaver combines an evidence-linked Hazard Knowledge Compiler with a Hazard Capability Graph that checks…
HyperBrowseComp: A Multilingual and Multimodal Stress Test for Web-Browsing Agents
The preprint presents 423 manually authored and human-validated questions across 13 languages. Items require obscure evidence discovery, multi-step clue chains, and heterogeneous sources such as videos, scanned…
Recursive Harness Self-Improvement for Frontier Reasoning Data Synthesis
The preprint introduces task-harness co-evolution for recursive reasoning-data synthesis. Online self-improvement turns solver failures into reusable skills, while post-task updates revise skills, prompts, and…
Agents Are Systems, Not Models: Rethinking Agentic Evaluation
The preprint evaluates agents as configurable systems whose task information, reasoning settings, self-verification, time budget, and backbone model all affect outcomes. On four scientific tasks, the abstract reports…
Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval
The preprint identifies a retrieval-level positivity problem: memories that are never retrieved cannot have their utility identified by store-level interventions. It proposes Causal Memory Policy, which reserves…
Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks
The preprint argues that many failures of 2B–9B open models on real tasks arise from the harness rather than the model. Mingbird is a local-first harness for Windows and Ollama with mechanisms including a byte-level…
ScholarCatalyst: A Benchmark for Retrieving Papers That Inspire New Research
The preprint builds a benchmark from judgments by 184 lead authors of 207 recent computer-science papers, who identified prior papers that advanced or could have advanced their projects and provided rationales. The…
VISTA: A Visual Harness for Reasoning in an Interactive World
The technical report introduces a visual harness that gives a multimodal model long-horizon visual context through direct perception, lossless visual memory, active retrieval, and input reorganization. The abstract…
The AI Assessment Sandbox Configurator: A Framework to Support Technical Assessment in AI Regulatory Sandboxes
The preprint describes an open-source framework for technical testing in AI regulatory sandboxes, motivated by EU AI Act obligations. It proposes a plug-in test and control catalogue, a shared data model for…
External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing
The preprint presents a hidden-state method for identifying the onset and continuation of hallucinated spans rather than making only token-level binary judgments. It proposes an external observer model that probes the…
Homomorphic Advantage Operator: Stabilizing Reinforcement Learning Under Fully Homomorphic Encryption Constraints
The preprint proposes the Homomorphic Advantage Operator to stabilize deep reinforcement learning when nonlinear operations are approximated under fully homomorphic encryption. It applies a zero-mean centering…
Mimir: Physics-Grounded LLM Agents for Long-Horizon Irrigation Control
The preprint studies long-running irrigation decisions where actions change future states and errors compound. Mimir combines a structured physical interface, deterministic simulation, verified revision, bounded action…
OpenMTB-Audit: Exposing Over-Refusal and Clinical Expert Perspectives in LLM-Based Molecular Tumor Board Safety Evaluation
The preprint introduces a benchmark of 500 synthetic non-small-cell lung-cancer cases with four safety labels, including Supported, Partially Supported, Unsupported, and Insufficient Information. Across eight model…
Page 1 of 47
