Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
52 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on safety
52 notes, page 1 of 3
Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs
Across 18 base-instruct model pairs spanning Qwen, Llama, and Gemma, the authors report that harmfulness is more readily decoded from attention updates while MLP updates contribute more strongly to harm-selective…
DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design
DRYLAB-Bench evaluates agentic mutation-design capability in 30 hazardous scenarios distilled from biosecurity guidelines, using multi-round tool-augmented interactions. It scores designs against 187,695 empirical…
X-AgentRed: Red-Teaming Personal Agents in Evolving Multi-Service Ecosystems
X-AgentRed constructs a directed graph of tool dependencies, effects, harms, and execution preconditions to discover harmful cross-service action paths. Execution feedback expands and refines the paths through tool…
Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
The preprint evaluates epistemic humility through three trajectory-level behaviors: Identify, Solve, and Escalate. It tests controlled and naturally occurring conflicts between parametric knowledge and retrieved…
OnTrack: Real-Time Monitoring and Intervention in LLM Agent Trajectories via Streaming Structure-Aware Optimal Transport
The preprint proposes OnTrack, a streaming monitoring mechanism that compares an agent's steps and dependencies against recorded successful runs to alert users or block the agent. It evaluates full-reference…
Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoff
The preprint develops a population-level model in which agent fitness depends on cybersecurity capability. It reports that without collaboration, takeoff requires individual capability above a critical threshold; with…
Searching for "Harmful Refusal": A Psychometric Audit of an AI Safety Benchmark
The preprint audits whether harmful refusal is a single measurable attribute in HELM Safety. It reports saturation in three candidate datasets and uses multidimensional item-response theory plus differential item…
Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness
The preprint tests whether legal models' named statutes or precedents causally support their verdicts. Holding case facts fixed, it substitutes the named authority and examines verdict changes and hidden-state dynamics…
Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems
The preprint tests regulatory-rule sensitivity by deleting, swapping, or negating a rule while holding the case fixed across five models and 20 domains. It reports that verdicts are often invariant to large rule…
Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations
The preprint presents Transect, an open-source package built on Inspect Scout for analyzing long agent runs whose transcripts can span hundreds of pages. Configurable evaluation families align recorded events, token…
VeriFine: Scaling Verification for Self-Improvement in Embodied Reasoning
The preprint proposes an agent harness that co-evolves the policy, training curriculum, and judge. A Policy Improvement Loop diagnoses recurring failures and builds an adaptive curriculum; when verification becomes a…
ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding
The preprint introduces a benchmark for unified evaluation of risk-treatment capabilities in coding agents. It maps Avoidance, Transfer, Mitigation, and Acceptance from software risk management to 200…
AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy
The preprint introduces AgentPrivArena, a reproducible environment that integrates authentic MCP tools and self-hosted services to evaluate privacy risks during realistic multi-step agent execution. It proposes…
What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents
The preprint introduces AgentMonBench, a software-engineering benchmark for judging requirement/behavior alignment and identifying consequential autonomous decisions that warrant verification. Its Evidence-Grounded…
HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention
The preprint introduces HERA, a framework that constructs verifiable feasible/infeasible task pairs through controlled environment mutations and adapts the agent harness toward prior weaknesses. The abstract reports…
HazardWeaver: Scientific Route Selection for Hazard Analysis Agents
The preprint formulates hazard-analysis automation as state-dependent scientific route selection. HazardWeaver combines an evidence-linked Hazard Knowledge Compiler with a Hazard Capability Graph that checks…
External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing
The preprint presents a hidden-state method for identifying the onset and continuation of hallucinated spans rather than making only token-level binary judgments. It proposes an external observer model that probes the…
Mimir: Physics-Grounded LLM Agents for Long-Horizon Irrigation Control
The preprint studies long-running irrigation decisions where actions change future states and errors compound. Mimir combines a structured physical interface, deterministic simulation, verified revision, bounded action…
OpenMTB-Audit: Exposing Over-Refusal and Clinical Expert Perspectives in LLM-Based Molecular Tumor Board Safety Evaluation
The preprint introduces a benchmark of 500 synthetic non-small-cell lung-cancer cases with four safety labels, including Supported, Partially Supported, Unsupported, and Insufficient Information. Across eight model…
Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training
The preprint introduces 50,228 error-diagnosis pairs from 9,961 tasks across 33 environments, 19 harness families, and 23 policy models. Its Agentic Error-to-Training pipeline preserves traces and execution metadata…
HARISSA: Inference-Time Self-Checks for Efficient and Safe Local Language Model Deployment
The preprint uses hidden states before generation and at the end of an answer to predict correctness for local-model deployment. HARISSA fine-tunes both states and uses a single policy to choose among increasingly…
Failure-Transparent Agents: Benchmarking Post-Failure Reporting in Tool-Using Language Models
The preprint introduces a controlled benchmark for cases where a required tool fails and an agent may report success without supporting evidence. The benchmark contains 100 tasks with deterministic failure traces…
How does Adversarial Influence Scale in Multi-Agent Systems?
The preprint studies deception in multi-agent deliberation. Its abstract reports that susceptibility depends on the proportion of deceivers rather than total group size, that defection rises approximately linearly with…
PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety
PASTABench evaluates safety monitoring over multi-turn agent trajectories, covering 1,139 trajectories, five risk categories, and 13 subcategories. It proposes an Optimal Intervention Window based on the earliest…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
