Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Item Response Theory for AI Safety

Source

arXiv

arxiv.org/abs/2608.05086v1

Read the source

Source

Plain-Language Brief

Researchers used Item Response Theory (IRT), a statistical toolkit, to analyze the safety performance of 192 language models across eight different safety benchmarks. The study found that three key factors - refusal strictness, truthfulness, and contextual harm - explain most of the variation between models. IRT helped in reducing evaluation cost by 97-99% by using adaptively chosen items instead of random subsets, and it can be used to detect naive sandbagging and changes of models behind APIs.

What the Document Says

  • The research focuses on measuring the safety behavior of language models and uses Item Response Theory (IRT) for this purpose.
  • Eight safety benchmarks were analyzed across 192 language models, making it the largest psychometric analysis of LLM safety evaluations to date.
  • Three factors - refusal strictness, truthfulness, and contextual harm - explain most of the variance between models across benchmarks.
  • Psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size.
  • Adaptively chosen items can reduce evaluation cost by 97-99%.
  • IRT supports audits of individual models, helping to detect naive sandbagging and changes of model behind APIs.
  • The authors recommend that frontier labs and evaluators adopt IRT for reading, reducing, and auditing safety benchmarks.

Key Concepts Mentioned

  • Language Models
  • Item Response Theory (IRT)
  • Safety Benchmarks
  • Refusal Strictness
  • Truthfulness
  • Contextual Harm

Claims Made by the Document

  • Three factors explain most of the variance between models across benchmarks.
  • Psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size.
  • Adaptively chosen items can reduce evaluation cost by 97-99%.
  • IRT supports audits of individual models.

Evidence Used

  • The study analyzed 192 language models across eight safety benchmarks.
  • The results show that three factors explain most of the variance between models.
  • Psychometrically selected items were found to recover full benchmark scores with lower error.
  • Adaptively chosen items were found to reduce evaluation cost by 97-99%.
  • IRT was used to support audits of individual models in the study.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because benchmarks duplicate one another, correlate heavily, and models may sandbag when they detect evaluation. To address these issues, we draw on Item Response Theory (IRT), a statistical toolkit for measuring these latents from performance on items with inferred psychometric properties. We fit IRT models to eight safety benchmarks across 192 language models, the largest psychometric analysis of LLM safety evaluations to date, and contribute three results. First, we find that three interpretable factors of refusal strictness, truthfulness, and contextual harm explain most of the variance between models across benchmarks. Second, psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size, and roughly ten adaptively chosen items suffice for several individual benchmarks, cutting evaluation cost by 97-99%. Third, IRT supports audits of individual models, showing that it can be used to detect naive sandbagging and changes of model behind APIs. Overall, we show IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.