Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Item Response Theory for AI Safety

Source

arXiv

arxiv.org/abs/2608.05086v1

Read the source

Source

Plain-Language Brief

The paper proposes using Item Response Theory (IRT) to measure the safety of language models. IRT is a statistical method used to analyze the latent traits of individuals from their performance on specific tasks. The authors apply IRT to eight safety benchmarks for 192 language models, finding that three factors - refusal strictness, truthfulness, and contextual harm - explain most of the variance between models across benchmarks. They also demonstrate that psychometrically selected items can recover full benchmark scores with lower error than random subsets, and that IRT can be used to detect naive sandbagging and changes in models behind APIs.

What the Document Says

  • The paper discusses the use of Item Response Theory (IRT) for measuring the safety of language models.
  • Eight safety benchmarks were analyzed across 192 language models using IRT.
  • Three factors - refusal strictness, truthfulness, and contextual harm - explain most of the variance between models across benchmarks.
  • Psychometrically selected items can recover full benchmark scores with lower error than random subsets.
  • Approximately ten adaptively chosen items are sufficient for several individual benchmarks, reducing evaluation cost by 97-99%.
  • IRT supports audits of individual models, enabling the detection of naive sandbagging and changes in models behind APIs.

Key Concepts Mentioned

  • Item Response Theory (IRT)
  • language models
  • safety benchmarks
  • refusal strictness
  • truthfulness
  • contextual harm

Claims Made by the Document

  • IRT can be used to measure the safety of language models.
  • Three factors explain most of the variance between models across benchmarks.
  • Psychometrically selected items can recover full benchmark scores with lower error than random subsets.
  • Approximately ten adaptively chosen items are sufficient for several individual benchmarks.
  • IRT can be used to detect naive sandbagging and changes in models behind APIs.

Evidence Used

  • The paper applies IRT to eight safety benchmarks for 192 language models.
  • The results show that three factors explain most of the variance between models across benchmarks.
  • The results demonstrate that psychometrically selected items can recover full benchmark scores with lower error than random subsets.
  • The results indicate that approximately ten adaptively chosen items are sufficient for several individual benchmarks.
  • The paper presents examples showing that IRT can be used to detect naive sandbagging and changes in models behind APIs.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because benchmarks duplicate one another, correlate heavily, and models may sandbag when they detect evaluation. To address these issues, we draw on Item Response Theory (IRT), a statistical toolkit for measuring these latents from performance on items with inferred psychometric properties. We fit IRT models to eight safety benchmarks across 192 language models, the largest psychometric analysis of LLM safety evaluations to date, and contribute three results. First, we find that three interpretable factors of refusal strictness, truthfulness, and contextual harm explain most of the variance between models across benchmarks. Second, psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size, and roughly ten adaptively chosen items suffice for several individual benchmarks, cutting evaluation cost by 97-99%. Third, IRT supports audits of individual models, showing that it can be used to detect naive sandbagging and changes of model behind APIs. Overall, we show IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.