Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
5 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on safety evaluation
5 notes
Item Response Theory for AI Safety
The paper presents a study using Item Response Theory (IRT) to analyze the safety of language models. The authors find that three factors - refusal strictness, truthfulness, and contextual harm - explain most of the…
Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levels
The research proposes JADR, a protocol for measuring the internal safety mechanisms of language models before their first response is generated. It uses Jacobian space to record top-k J-space tokens from prompts and…
CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis
Researchers have proposed a new method called CARLA-GS for generating corner cases in autonomous driving simulations that are essential for safety evaluation. The method decouples the visual representation, semantic…
Whose Side Is Your Agent On? Multi-Party Principal Loyalty in LLM Agents
In a rapidly growing field of artificial intelligence agents, there is a need to ensure that these agents remain loyal to the principal they represent while also engaging with a counterparty whose interests may…
Do Encoders Suffice? A Systematic Comparison of Encoder and Decoder Safety Judges for LLM Adversarial Evaluation
This research compares the effectiveness of encoder classifiers (using models like ModernBERT and Ettin) against LLM-based judges in identifying harmful outputs from large language models (LLMs). The study evaluates…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
