Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
41 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on ai safety
41 notes, page 1 of 2
How to Verify Consistency of Probabilistic Claims
Researchers propose an interactive protocol that allows a polynomial-time verifier to check the approximate consistency of a probabilistic predictor's answers in polynomial time. The protocol is relevant for AI safety…
Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy
The authors propose a system that uses large language models (LLMs) to evaluate the permissibility of actions planned by robots before they are executed. The system aims to reduce risks associated with biased planning…
Multi-Agent AI Safety as an Institutional Design Problem
The paper discusses AI safety as an institutional design problem in multi-agent systems. The research focuses on understanding which parts of an AI institution ensure safety and how they do it. The study involves a…
Multi-Agent AI Safety as an Institutional Design Problem
The paper discusses an experiment on multi-agent AI safety as an institutional design problem. The research focuses on understanding which parts of an AI institution ensure safety and how they do it. The study includes…
SHE: Trajectory-driven Safety Harness Evolution for LLM Agents
Researchers propose a framework called Safety Harness Evolution (SHE) for improving the safety of large language model agents. SHE focuses on evolving the harness, which manages context, memory, tools, permissions, and…
SHE: Trajectory-driven Safety Harness Evolution for LLM Agents
Researchers have proposed a framework called Safety Harness Evolution (SHE) for large language model (LLM) agents. SHE aims to improve the safety of these agents by learning evolving safe boundaries from rollout…
SHE: Trajectory-driven Safety Harness Evolution for LLM Agents
Researchers propose a framework called Safety Harness Evolution (SHE) to improve the safety of large language model agents. SHE learns evolving safe boundaries from rollout trajectories, allowing for localized harness…
Resourced Authority A Mechanism-Design Model for Participatory Governance of Deployed AI Agents
The authors propose a model for continuous participatory governance of AI agents, focusing on controlling them through resource allocation to make authorization self-enforcing via compute budgets. The model consists of…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The study audits the assumptions made in large language model (LLM) benchmark evaluations, focusing on one popular LLM - ChatGPT. The research compares two access modalities (chat UI and API), with and without web…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The research examines the limitations of current benchmarks used to evaluate large language models (LLMs) for safety, reliability, and deployment readiness. The study focuses on a popular LLM, ChatGPT, comparing its…
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
The study examines the limitations of current AI benchmarks by focusing on factors such as modality (chat UI vs API), search capabilities, consistency of responses, citation grounding, and abstention behavior using…
Item Response Theory for AI Safety
The paper proposes using Item Response Theory (IRT) to measure the safety of language models. IRT is a statistical method used to analyze the latent traits of individuals from their performance on specific tasks. The…
Item Response Theory for AI Safety
Researchers used Item Response Theory (IRT), a statistical toolkit, to analyze the safety performance of 192 language models across eight different safety benchmarks. The study found that three key factors - refusal…
On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment
A research paper proposes a novel method called Routing-based On-Policy Distillation (ROPD) to improve the safety of large language models (LLMs). The current fine-tuning approach has a vulnerability where harmful…
Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls
Researchers propose a new method for controlling risk in language model agents' actions, which are performed through structured tool calls. The proposed method, called role-stratified per-field conformal risk control…
Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls
Researchers introduce a new method called role-stratified per-field conformal risk control for language model agents that perform actions through structured tool calls. This method aims to ensure safety by setting…
Epistemic Norms for AI Safety and Alignment Research
The text discusses a new epistemic code named ECAISA for AI Safety and Alignment Research, which aims to ensure that catastrophic failures in AI never occur under specific conditions. The code focuses on two main…
Same Dangerous Objective, Opposite Advice: Direct Exposure versus Multi-Agent Mediation
A study found that a high-capacity language model (gpt-5.6-sol) behaves differently when presented with dangerous objectives directly compared to when other agents manipulate and relay the instructions. The model gave…
Same Dangerous Objective, Opposite Advice: Direct Exposure versus Multi-Agent Mediation
A study using OpenAI's gpt-5.6-sol model found that the model gave different advice when presented with a dangerous objective directly compared to when other agents transformed and relayed the direction. When shown an…
Same Dangerous Objective, Opposite Advice: Direct Exposure versus Multi-Agent Mediation
A study comparing direct exposure to a dangerous objective versus multi-agent mediation using OpenAI's gpt-5.6-sol model reveals that direct exposure leads to advice contrary to the target, while mediated exposure…
Same Dangerous Objective, Opposite Advice: Direct Exposure versus Multi-Agent Mediation
A study compares the behavior of a high-capability language model when it is given dangerous objectives directly versus through other agents. The results show that the model gives advice against its target when given…
Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs
A study investigates the impact of a structured evidence-sufficiency prompt on reducing overconfident answers from four language models in a clinical context. The study finds a significant reduction in unsafe…
Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs
A study investigates the impact of a structured evidence-sufficiency prompt on reducing unsafe overconfident answers from four different language models in a medical context. The study finds that the prompt…
Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming
Researchers have developed a new method called AHA for automated red-teaming of production language models like Claude Code and Codex. This method uses one agentic research environment to discover reusable…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
