Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

Source

arXiv

arxiv.org/abs/2607.07474v1

Read the source

Source

Plain-Language Brief

The paper introduces an action-graded harm rubric for evaluating AI agents in a red-teaming scenario. This new method rates the severity of an agent's actions on a seven-level scale, considering factors such as reversibility, scope crossing, and privilege expansion. The evaluation is done by both a deterministic oracle and a panel of three language model judges.

What the Document Says

  • The paper discusses the limitations of current agentic red-teaming benchmarks that only report whether an attack was successful or not.
  • It proposes an action-graded harm rubric that scores an agent's tool-call trajectory on a seven-level ordinal scale (L0 to L6) based on reversibility, scope crossing, and privilege expansion.
  • The severity grading is performed using a deterministic oracle and a panel of three frontier language-model judges.
  • The evaluation is conducted across four victim models and two defenses on the AgentDojo workspace suite.
  • Three cases hidden by the binary metric are exposed, including a defense allowing an externally visible cross-scope leak through an unfiltered tool.
  • The judge panel shows high ordinal agreement with the oracle (Krippendorff's alpha = 0.91) but has systematic blind spots, particularly failing to recognize escalation chains.
  • The contribution is a reusable, trace-grounded severity instrument applied to existing red-team logs.

Key Concepts Mentioned

  • agentic red-teaming
  • action-graded harm rubric
  • ordinal scale
  • reversibility
  • scope crossing
  • privilege expansion
  • deterministic oracle
  • language model judges
  • victim models
  • defenses
  • AgentDojo workspace suite
  • Krippendorff's alpha

Claims Made by the Document

  • The current agentic red-teaming benchmarks discard the information a defender needs most.
  • The proposed action-graded harm rubric provides a more comprehensive evaluation of an AI agent's actions.

Evidence Used

  • Binary attack-success rate discards the information a defender needs
  • Three cases hidden by the binary metric are exposed
  • High ordinal agreement between the oracle and judge panel

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not. We argue that this binary attack-success rate discards the information a defender most needs, namely how harmful the resulting action was. We introduce an action-graded harm rubric that scores an agent's tool-call trajectory on a seven-level ordinal scale (L0 to L6) according to whether the executed action was reversible, whether it crossed scope to reach another party, and whether it expanded privilege. We compute the scale two ways: a deterministic oracle that reads the trajectory and the attacker's stated goal, and a panel of three frontier language-model judges that read a tag-free account of the same trajectory. Across four victim models and two defenses on the AgentDojo workspace suite, severity grading exposes three cases the binary metric hides, including a defense that reports a zero attack-success rate while still permitting an externally visible cross-scope leak through an unfiltered tool. The judge panel reproduces the oracle with high ordinal agreement (Krippendorff's alpha = 0.91) but shares systematic blind spots that we characterize, most notably a failure to recognize escalation chains. Unlike prior work that provides harm taxonomies, harmful-task completion tests, execution-level safety benchmarks, or severity-aware simulation, our contribution is a reusable, trace-grounded severity instrument applied to the actual actions recorded in existing red-team logs. All code, prompts, and per-episode logs are released.