Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

Source

arXiv

arxiv.org/abs/2607.18063v1

Read the source

Source

Plain-Language Brief

The paper presents a new benchmark for testing the security of large language model (LLM) agents called 'Adaptive Adversaries'. The benchmark involves 21 scenarios where an adaptive attacker can observe prior defender responses and pivot across rounds, while each defender response is evaluated as a fresh interaction. The study finds that the success rate of adaptive attacks increases significantly with more rounds, and using multiple frontier attacker LLMs uncovers more unique successful attacks than a single attacker. The authors compare the performance of Claude Opus 4.6 and GPT-5.4 and find differences in their weaknesses. They also provide data and resources for the benchmark, including transcripts, an attack-replay dataset, and battle results from an open competition.

What the Document Says

  • A new benchmark called 'Adaptive Adversaries' is presented for testing the security of LLM agents.
  • The benchmark involves 21 scenarios where an adaptive attacker can observe prior defender responses and pivot across rounds.
  • Each defender response is evaluated as a fresh interaction.
  • Restricting scoring to the first attacker turn yields a low attack success rate (ASR).
  • Allowing 15 rounds of adaptive attack increases ASR to 5.4-14.0%.
  • Pooling three frontier attacker LLMs uncovers more unique successful attacks than a single attacker.
  • The generated attacks have low cosine similarity to attacks in existing benchmarks.
  • Claude Opus 4.6 and GPT-5.4 perform similarly in aggregate (5.4% each), but their weaknesses differ sharply.
  • On one scenario, Opus reaches a high ASR while GPT-5.4 and Gemini stay at a lower ASR.
  • The study finds that 13 out of 21 scenarios distinguish at least one defender pair, yet rankings disagree across scenarios.
  • The authors provide data and resources for the benchmark.

Key Concepts Mentioned

  • Adaptive Adversaries
  • LLM agents
  • adaptive attacker
  • defender responses
  • fresh interaction
  • attack success rate (ASR)
  • frontier attacker LLMs
  • Claude Opus 4.6
  • GPT-5.4
  • Gemini
  • Kendall's W

Claims Made by the Document

  • The benchmark is effective in testing the security of LLM agents.
  • Adaptive attacks are more successful with more rounds.
  • Using multiple frontier attacker LLMs uncovers more unique successful attacks.

Evidence Used

  • The study presents a new benchmark for evaluating LLM agent security.
  • The study finds that restricting scoring to the first attacker turn yields low ASR, and allowing more rounds increases ASR.
  • The study compares the performance of multiple frontier attacker LLMs.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn. We present a 21-scenario benchmark for \emph{adaptive multi-round attacks against memoryless LLM defenders}: an autonomous LLM attacker observes prior defender responses and pivots across rounds, while each defender response is evaluated as a fresh interaction. Holding the 21 scenarios, attackers, defenders, and structured-output scoring fixed, restricting scoring to the first attacker turn yields $0$-$1%$ attack success rate (ASR); allowing 15 rounds of adaptive attack yields $5.4$-$14.0%$. Pooling three frontier attacker LLMs uncovers $1.4$-$2.2\times$ as many unique successful attacks as the best single attacker, and the generated attacks have low cosine similarity ($0.02$-$0.14$) to attacks in existing benchmarks. Claude Opus 4.6 and GPT-5.4 are tied in aggregate ($5.4%$ each; overlapping $95%$ CIs), but their weaknesses differ sharply: on one scenario Opus reaches $60%$ ASR ($95%$ CI $36$--$80%$) while GPT-5.4 and Gemini each stay at $7%$ (CI $1$-$30%$; the gap is preserved in a higher-$N$ replication). $13$ of $21$ scenarios distinguish at least one defender pair, yet rankings disagree across scenarios (Kendall's $W = 0.19$). We release the benchmark -- 21 evaluation scenarios, 10 public development scenarios, the orchestrator, baseline harnesses, and a multi-attacker CLI -- plus 945 transcripts from the 3$\times$3 frontier matrix, an attack-replay dataset, and 18{,}422 gpt-oss-20b battles from an open competition's final scoring rounds.