Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failures

Source

arXiv

arxiv.org/abs/2606.30219v1

Read the source

Source

Plain-Language Brief

This paper presents EvalSafetyGap, a hybrid approach combining survey and conceptual framework for evaluating the safety of Large Language Models (LLMs). The authors aim to address the shared challenge in LLM evaluation and AI safety where scores and metrics can improve without verifying the underlying properties. The research covers eight evidence streams from 2018-2026, introducing an Instability Decomposition and an Alignment Trilemma as tools for comparison. The study audits ten models, revealing that the association between capability and sustained adversarial robustness is statistically indeterminate in this sample. The research aims to support dynamic evaluation, transparent reporting, multi-attempt safety measurement, and auditable alignment practice.

What the Document Says

  • The paper introduces EvalSafetyGap, a hybrid approach that combines a systematic search with narrative synthesis and grey evidence for evaluating the safety of Large Language Models (LLMs).
  • It covers eight evidence streams from 2018-2026.
  • The research includes an Instability Decomposition and an Alignment Trilemma as tools for comparison.
  • An audit of ten models is conducted, with results showing that the association between capability and sustained adversarial robustness is statistically indeterminate in this sample.
  • The study suggests that the apparent open-closed safety gap is modest, driven mainly by governance and disclosure rather than behavioral robustness.

Key Concepts Mentioned

  • Large Language Models (LLMs)
  • EvalSafetyGap
  • Instability Decomposition
  • Alignment Trilemma
  • dynamic evaluation
  • transparent source reporting
  • multi-attempt safety measurement
  • auditable alignment practice

Claims Made by the Document

  • Improvements in benchmark scores, reward-model signals, and reported safety metrics can occur without verifying the underlying properties.
  • The association between capability and sustained adversarial robustness is statistically indeterminate in this sample.

Evidence Used

  • Benchmark validity, dynamic evaluation, LLM-as-judge reliability, safety evaluation, jailbreak/refusal robustness, reward hacking, mechanistic interpretability, and governance/auditability evidence are covered in the research.

Methods / Approach

  • Hybrid survey combining systematic search, narrative synthesis, and grey evidence
  • Ten-model audit

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

LLM evaluation and AI safety face a shared measurement problem: benchmark scores, reward-model signals, and reported safety metrics can improve while the latent properties they are meant to represent remain difficult to verify. This paper combines a hybrid survey - a systematic search paired with narrative synthesis and separately tracked grey evidence - with a conceptual framework and a structured ten-model audit. The synthesis spans eight evidence streams: benchmark validity, dynamic evaluation, LLM-as-judge reliability, safety evaluation, jailbreak/refusal robustness, reward hacking, mechanistic interpretability, and governance/auditability, covering 2018-2026 evaluation-safety measurement work. We introduce EvalSafetyGap as an organizing hypothesis for comparing evaluation-side and alignment-side proxy failures under optimization pressure, using Goodhart's Law together with two constructs we develop here - an Instability Decomposition and an Alignment Trilemma - as tools for generating testable comparisons. The audit shows how conclusions shift when capability, behavioral safety, and governance are measured separately. In this sample (n = 10), the association between capability and sustained adversarial robustness is statistically indeterminate using the displayed Table 3 inputs (Pearson r = +0.232, p = 0.520), and the apparent open-closed safety gap is modest, driven mainly by governance and disclosure rather than behavioral robustness, and sensitive to how a single borderline model is classified; attempt-budget results are protocol dependent. Because the public evidence uses heterogeneous protocols, the audit is diagnostic rather than rank-generating. The contribution is a shared vocabulary and evidence map to support dynamic evaluation, transparent source reporting, multi-attempt safety measurement, and auditable alignment practice.