Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
100 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on agents
100 notes, page 1 of 5
DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design
DRYLAB-Bench evaluates agentic mutation-design capability in 30 hazardous scenarios distilled from biosecurity guidelines, using multi-round tool-augmented interactions. It scores designs against 187,695 empirical…
A Privileged Teacher That Only Scores Cannot Search: Critique-Conditioned On-Policy Self-Distillation for Reasoning
The study tests whether scoring signals in on-policy self-distillation localize errors and push probability away from erroneous student tokens. It reports that full-distribution KL and token log-ratio signals do…
RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
RATED combines a frozen dual encoder for semantic matching with a lightweight contextual-bandit module that reranks tools using real-time execution feedback. The authors introduce EcoAgent, a benchmark of 5,172 tools…
X-AgentRed: Red-Teaming Personal Agents in Evolving Multi-Service Ecosystems
X-AgentRed constructs a directed graph of tool dependencies, effects, harms, and execution preconditions to discover harmful cross-service action paths. Execution feedback expands and refines the paths through tool…
BrickBench: Evaluating Agentic Brick Design
The preprint introduces BrickBench, a benchmark for text-conditioned LEGO-set design. Agents must produce assemblies that satisfy semantic and design criteria while remaining physically buildable, selecting parts from…
Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
The preprint evaluates epistemic humility through three trajectory-level behaviors: Identify, Solve, and Escalate. It tests controlled and naturally occurring conflicts between parametric knowledge and retrieved…
Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents
The preprint proposes Hippocam, a hierarchical memory and continual-learning architecture that organizes ongoing work as nested intents. Active context stays centered on the current intent, while completed intents are…
OnTrack: Real-Time Monitoring and Intervention in LLM Agent Trajectories via Streaming Structure-Aware Optimal Transport
The preprint proposes OnTrack, a streaming monitoring mechanism that compares an agent's steps and dependencies against recorded successful runs to alert users or block the agent. It evaluates full-reference…
When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation
The preprint proposes Reasoning Adaptation through Cross-Turn Estimation (RACE), which estimates when prior reasoning remains sufficient for subsequent actions. Its LoGiC procedure identifies reasoning turns whose…
Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoff
The preprint develops a population-level model in which agent fitness depends on cybersecurity capability. It reports that without collaboration, takeoff requires individual capability above a critical threshold; with…
On the estimation and validity of AI time horizons---a statistical look at the METR plot
The preprint recomputes METR-style 50% AI time horizons over 228 tasks and 26 AIs using splines and item-response theory. The authors report that the fitted spline is nearly flat from roughly 2–30 minutes and close to…
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution
EvoAlloc is described as a self-evolving resource-allocation agent for LLM-based program evolution. It consolidates prior search and allocation outcomes into reusable experience, revises its allocation strategy, and…
Recursive Self-Improvement through Multi-Agent Self-Supervision
The work describes MASS, which alternates evolutionary workflow optimization with supervised fine-tuning on self-generated trajectories. The abstract reports discovery of multi-agent workflows and, over two cycles with…
Prior or Feedback? What an LLM Uses When Adapting Neural Operators
The study tests whether an LLM scientific agent adapts neural-operator tuning using experimental feedback under a limited trial budget. The authors report lower held-out nRMSE than random search and Bayesian…
nanoMuse: An Open-Source Personal Agent for Every Device You Own
The preprint presents nanoMuse as an open-source counterpart to persistent personal agents that operate across a user's accounts and devices. It describes a shared conversation over a user-runnable relay, a Sentinel…
ScienceClaw: Benchmarking Continual Self-Evolution of AI-for-Science Agents Across the Natural and Social Sciences
The preprint formalizes fixed-parameter program self-evolution that links task solving, scientific verification, and program updates. ScienceClaw-Eval spans 23 disciplines and measures correctness, evolutionary gain…
Sherpa: Teaching LLMs to Teach Adaptively
The preprint introduces Sherpa, a multi-turn reinforcement-learning framework that trains an LLM teacher against multiple student archetypes with distinct learning preferences. The teacher is optimized directly against…
Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations
The preprint presents Transect, an open-source package built on Inspect Scout for analyzing long agent runs whose transcripts can span hundreds of pages. Configurable evaluation families align recorded events, token…
Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?
The preprint introduces BOTTLED, a benchmark in which agents receive an unlabelled workload and must complete it under fixed time, compute, and LLM API budgets. Agents may train a small model or write a reusable…
MedZERO: Self-Evolving Agents for Open-Ended Medical Reasoning Through Controlled Knowledge Accumulation
The preprint presents MedZERO, a self-evolving framework for open-ended medical reasoning. An Examiner generates medical question-option pairs, while a Reasoner solves them with evidence-grounded multi-turn reasoning…
ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding
The preprint introduces a benchmark for unified evaluation of risk-treatment capabilities in coding agents. It maps Avoidance, Transfer, Mitigation, and Acceptance from software risk management to 200…
SquidAgent: Parallelize Wisely, Coordinate Efficiently
The preprint identifies re-exploration and alignment as hidden costs that can make parallel multi-agent systems slower than a single-agent baseline. It proposes a token-based decision criterion, direct forking from the…
WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?
The preprint introduces WorldSolver, a benchmark of 168 simulation tasks derived from physical phenomena in 61 classic computer-graphics papers across seven physical domains. Each task provides a code scaffold while…
AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy
The preprint introduces AgentPrivArena, a reproducible environment that integrates authentic MCP tools and self-hosted services to evaluate privacy risks during realistic multi-step agent execution. It proposes…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
