Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)

Source

arXiv

arxiv.org/abs/2608.06202v1

Read the source

Source

Plain-Language Brief

The study audits the assumptions made in large language model (LLM) benchmark evaluations, focusing on one popular LLM - ChatGPT. The research compares two access modalities (chat UI and API), with and without web search enabled, using a total of 401 prompts from two benchmarks (BBQ and SafetyBench). The study finds that chat UI responses are less accurate than API responses with search disabled, enabling web search reduces accuracy, repeated runs produce inconsistent responses, the two modalities ground answers in different citations, and abstention behavior is inconsistent. The research suggests that AI safety evaluations should consider modality, multi-run consistency, search conditions, and response-level behaviors to better reflect how deployed AI systems behave in practice.

What the Document Says

  • The study examines large language model (LLM) benchmark evaluations, specifically focusing on ChatGPT.
  • Two modalities are compared: chat UI and OpenAI's API.
  • Web search is enabled or disabled in the comparison.
  • The research uses 401 prompts from BBQ and SafetyBench benchmarks, resulting in 4812 responses across three repeated runs per prompt.
  • Beyond standard performance measures, the study evaluates response consistency, response text similarity, citation grounding, and abstention behavior.
  • Chat UI responses were less accurate than API responses with search disabled on both benchmarks.
  • Enabling web search reduced accuracy by up to 8 percentage points.
  • Repeated runs of the same prompt produced inconsistent responses in up to 21% of prompts.
  • The two modalities grounded answers in different citations.
  • Abstention behavior was also inconsistent across both modalities.

Key Concepts Mentioned

  • Large Language Models
  • ChatGPT
  • OpenAI API
  • Web Search
  • BBQ Benchmark
  • SafetyBench Benchmark
  • Modality
  • Response Consistency
  • Response Text Similarity
  • Citation Grounding
  • Abstention Behavior

Claims Made by the Document

  • Chat UI responses are less accurate than API responses with search disabled.
  • Enabling web search reduces accuracy.
  • Repeated runs of the same prompt produce inconsistent responses.
  • The two modalities ground answers in different citations.
  • Abstention behavior is inconsistent across both modalities.

Evidence Used

  • Comparison of chat UI and API responses, use of 401 prompts from BBQ and SafetyBench benchmarks, collection of 4812 total responses, evaluation of response consistency, response text similarity, citation grounding, and abstention behavior.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Large language model (LLM) benchmark evaluations are routinely used to support claims about model safety, reliability, and deployment readiness. Yet most evaluations rely on a single access modality (model APIs), perform a single run per prompt, and report accuracy as the primary outcome metric, without accounting for conditions such as web search that may have effects on model behavior in deployment. We audit these assumptions for one of the most widely-used LLMs, comparing two modalities, ChatGPT's chat UI and OpenAI's API, with and without web search enabled. We use a stratified total sample of 401 prompts from two popular benchmarks, BBQ and SafetyBench, collecting 4,812 total responses across three repeated runs per prompt. Beyond standard performance measures, we evaluate model output dimensions including response consistency, response text similarity, citation grounding, and abstention behavior. For instance, chat UI responses were less accurate than API responses on both benchmarks with search disabled. Enabling web search reduced accuracy by up to 8 percentage points, and even reversed the direction of modality performance trends for one benchmark. Repeated runs of the same prompt produced inconsistent responses in up to 21% of prompts. The two modalities also grounded answers in different citations, and abstention behavior was also inconsistent across both modalities. These results illustrate that, even within a model family, reporting only simple accuracy metrics can obscure important forms of model behavioral variation relevant to AI safety assessments. We argue that AI safety evaluations should systematically account for modality, multi-run consistency, search conditions, and response-level behaviors to better reflect how deployed AI systems behave in practice.