Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

Source

arXiv

arxiv.org/abs/2607.18063v1

Read the source

Source

Plain-Language Brief

The paper presents a new benchmark for testing the security of LLM (Language Model) agents against adaptive adversaries in multi-turn interactions. The benchmark, called Adaptive Adversaries, evaluates how defenders respond to an attacker that adapts its approach based on prior defender responses over multiple rounds. The authors find that allowing the attacker 15 rounds of adaptive attacks increases the success rate significantly compared to just the first round. They also observe differences in performance between LLM models like Claude Opus 4.6 and GPT-5.4, with Opus showing a higher success rate on certain scenarios. The benchmark includes evaluation scenarios, development scenarios, an orchestrator, baseline harnesses, and other resources.

What the Document Says

  • The paper introduces a new benchmark called Adaptive Adversaries for testing the security of LLM agents in multi-turn interactions with adaptive adversaries.
  • The benchmark involves 21 evaluation scenarios, 10 public development scenarios, an orchestrator, baseline harnesses, and other resources.
  • In the benchmark, an attacker observes prior defender responses and pivots across rounds while each defender response is evaluated as a fresh interaction.
  • Restricting scoring to the first attacker turn yields a low attack success rate (ASR), but allowing 15 rounds of adaptive attack increases the ASR to 5.4-14.0%.
  • Pooling three frontier attacker LLMs uncovers more unique successful attacks than the best single attacker, and these attacks have low cosine similarity with attacks in existing benchmarks.
  • Claude Opus 4.6 and GPT-5.4 have different weaknesses: on one scenario, Opus reaches a higher ASR while GPT-5.4 and Gemini stay at a lower ASR.
  • 13 out of 21 scenarios distinguish at least one defender pair, but rankings disagree across scenarios.

Key Concepts Mentioned

  • LLM agents
  • adaptive adversaries
  • multi-turn interactions
  • benchmark
  • attack success rate
  • Claude Opus 4.6
  • GPT-5.4

Claims Made by the Document

  • Allowing the attacker 15 rounds of adaptive attacks increases the ASR significantly.
  • Pooling three frontier attacker LLMs uncovers more unique successful attacks than the best single attacker.
  • Claude Opus 4.6 and GPT-5.4 have different weaknesses.

Evidence Used

  • The text mentions that allowing 15 rounds of adaptive attack increases the ASR to 5.4-14.0%.
  • The text states that pooling three frontier attacker LLMs uncovers more unique successful attacks than the best single attacker.
  • The text notes differences in performance between Claude Opus 4.6 and GPT-5.4.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn. We present a 21-scenario benchmark for \emph{adaptive multi-round attacks against memoryless LLM defenders}: an autonomous LLM attacker observes prior defender responses and pivots across rounds, while each defender response is evaluated as a fresh interaction. Holding the 21 scenarios, attackers, defenders, and structured-output scoring fixed, restricting scoring to the first attacker turn yields $0$-$1%$ attack success rate (ASR); allowing 15 rounds of adaptive attack yields $5.4$-$14.0%$. Pooling three frontier attacker LLMs uncovers $1.4$-$2.2\times$ as many unique successful attacks as the best single attacker, and the generated attacks have low cosine similarity ($0.02$-$0.14$) to attacks in existing benchmarks. Claude Opus 4.6 and GPT-5.4 are tied in aggregate ($5.4%$ each; overlapping $95%$ CIs), but their weaknesses differ sharply: on one scenario Opus reaches $60%$ ASR ($95%$ CI $36$--$80%$) while GPT-5.4 and Gemini each stay at $7%$ (CI $1$-$30%$; the gap is preserved in a higher-$N$ replication). $13$ of $21$ scenarios distinguish at least one defender pair, yet rankings disagree across scenarios (Kendall's $W = 0.19$). We release the benchmark -- 21 evaluation scenarios, 10 public development scenarios, the orchestrator, baseline harnesses, and a multi-attacker CLI -- plus 945 transcripts from the 3$\times$3 frontier matrix, an attack-replay dataset, and 18{,}422 gpt-oss-20b battles from an open competition's final scoring rounds.