Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)

Source

arXiv

arxiv.org/abs/2608.06202v1

Read the source

Source

Plain-Language Brief

The research examines the limitations of current benchmarks used to evaluate large language models (LLMs) for safety, reliability, and deployment readiness. The study focuses on a popular LLM, ChatGPT, comparing its chat UI and API modalities with and without web search enabled. The authors found that using only accuracy as a metric can hide important variations in model behavior, such as inconsistent responses, different citation grounding, and abstention behavior across the two modalities. They suggest that AI safety evaluations should take into account modality, multi-run consistency, search conditions, and response-level behaviors for a more accurate reflection of how deployed AI systems function in practice.

What the Document Says

  • The study investigates the limitations of current benchmarks used to evaluate large language models (LLMs) for safety, reliability, and deployment readiness.
  • The research focuses on ChatGPT, comparing its chat UI and API modalities with and without web search enabled.
  • The authors found that accuracy is not a sufficient metric to reflect important variations in model behavior.
  • They observed inconsistent responses, different citation grounding, and abstention behavior across the two modalities.
  • The study argues that AI safety evaluations should account for modality, multi-run consistency, search conditions, and response-level behaviors.

Key Concepts Mentioned

  • large language models (LLMs)
  • ChatGPT
  • API
  • chat UI
  • web search
  • AI safety evaluations
  • modality
  • multi-run consistency
  • response-level behaviors

Claims Made by the Document

  • Using only accuracy as a metric can hide important variations in model behavior.
  • The two modalities (chat UI and API) grounded answers in different citations.
  • Abstention behavior was inconsistent across both modalities.

Evidence Used

  • Comparing two modalities of ChatGPT with and without web search enabled.
  • Collecting 4,812 total responses across three repeated runs per prompt from a stratified total sample of 401 prompts from two popular benchmarks (BBQ and SafetyBench).

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Large language model (LLM) benchmark evaluations are routinely used to support claims about model safety, reliability, and deployment readiness. Yet most evaluations rely on a single access modality (model APIs), perform a single run per prompt, and report accuracy as the primary outcome metric, without accounting for conditions such as web search that may have effects on model behavior in deployment. We audit these assumptions for one of the most widely-used LLMs, comparing two modalities, ChatGPT's chat UI and OpenAI's API, with and without web search enabled. We use a stratified total sample of 401 prompts from two popular benchmarks, BBQ and SafetyBench, collecting 4,812 total responses across three repeated runs per prompt. Beyond standard performance measures, we evaluate model output dimensions including response consistency, response text similarity, citation grounding, and abstention behavior. For instance, chat UI responses were less accurate than API responses on both benchmarks with search disabled. Enabling web search reduced accuracy by up to 8 percentage points, and even reversed the direction of modality performance trends for one benchmark. Repeated runs of the same prompt produced inconsistent responses in up to 21% of prompts. The two modalities also grounded answers in different citations, and abstention behavior was also inconsistent across both modalities. These results illustrate that, even within a model family, reporting only simple accuracy metrics can obscure important forms of model behavioral variation relevant to AI safety assessments. We argue that AI safety evaluations should systematically account for modality, multi-run consistency, search conditions, and response-level behaviors to better reflect how deployed AI systems behave in practice.