safety evaluation
5 notes, newest first.
Item Response Theory for AI Safety
The paper presents a study using Item Response Theory (IRT) to analyze the safety of language models. The authors find that three factors - refusal strictness, truthfulness, and contextual harm - explain most of the…
Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levels
The research proposes JADR, a protocol for measuring the internal safety mechanisms of language models before their first response is generated. It uses Jacobian space to record top-k J-space tokens from prompts and…
CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis
Researchers have proposed a new method called CARLA-GS for generating corner cases in autonomous driving simulations that are essential for safety evaluation. The method decouples the visual representation, semantic…
Whose Side Is Your Agent On? Multi-Party Principal Loyalty in LLM Agents
In a rapidly growing field of artificial intelligence agents, there is a need to ensure that these agents remain loyal to the principal they represent while also engaging with a counterparty whose interests may…
Do Encoders Suffice? A Systematic Comparison of Encoder and Decoder Safety Judges for LLM Adversarial Evaluation
This research compares the effectiveness of encoder classifiers (using models like ModernBERT and Ettin) against LLM-based judges in identifying harmful outputs from large language models (LLMs). The study evaluates…
