Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

Source

arXiv

arxiv.org/abs/2607.07474v1

Read the source

Source

Plain-Language Brief

The paper proposes an action-graded harm rubric for evaluating AI agents' behavior during cybersecurity attacks. This scale assesses the severity of an agent's actions based on factors like reversibility, scope crossing, and privilege expansion. The authors compare a deterministic oracle with human judges to grade the harm caused by AI agents across different models and defenses.

What the Document Says

  • The paper discusses Agnetic red-teaming benchmarks which determine whether an injected agent was compromised (attack succeeded) or not.
  • A new action-graded harm rubric is introduced, scoring AI agents' actions on a seven-level ordinal scale (L0 to L6) based on the executed action's reversibility, scope, and privilege.
  • Two methods for computing the scale are presented: a deterministic oracle reading the trajectory and attacker's goal, and a panel of three language model judges evaluating a tag-free account of the same trajectory.
  • The study is based on four victim models and two defenses from AgentDojo workspace suite.
  • Severity grading reveals cases that binary metric hides, such as a defense permitting an externally visible cross-scope leak through an unfiltered tool despite a zero attack-success rate.
  • The judge panel shows high agreement with the oracle (Krippendorff's alpha = 0.91) but also has blind spots, especially failure to recognize escalation chains.
  • The proposed approach is different from prior work providing harm taxonomies, harmful-task completion tests, execution-level safety benchmarks, or severity-aware simulation.

Key Concepts Mentioned

  • Agnetic red-teaming
  • Action-graded harm rubric
  • Deterministic oracle
  • Language model judges
  • Victim models
  • Defenses
  • AgentDojo workspace suite
  • Severity grading
  • Escalation chains

Claims Made by the Document

  • The binary attack-success rate discards the information a defender most needs.
  • The proposed action-graded harm rubric provides more useful information for defenders than the binary attack-success rate.
  • Severity grading exposes cases hidden by the binary metric.
  • The judge panel reproduces the oracle with high ordinal agreement.

Evidence Used

  • Binary attack-success rate discards information needed by defenders
  • Examples of cases hidden by binary metric revealed by severity grading
  • High agreement between judge panel and oracle

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not. We argue that this binary attack-success rate discards the information a defender most needs, namely how harmful the resulting action was. We introduce an action-graded harm rubric that scores an agent's tool-call trajectory on a seven-level ordinal scale (L0 to L6) according to whether the executed action was reversible, whether it crossed scope to reach another party, and whether it expanded privilege. We compute the scale two ways: a deterministic oracle that reads the trajectory and the attacker's stated goal, and a panel of three frontier language-model judges that read a tag-free account of the same trajectory. Across four victim models and two defenses on the AgentDojo workspace suite, severity grading exposes three cases the binary metric hides, including a defense that reports a zero attack-success rate while still permitting an externally visible cross-scope leak through an unfiltered tool. The judge panel reproduces the oracle with high ordinal agreement (Krippendorff's alpha = 0.91) but shares systematic blind spots that we characterize, most notably a failure to recognize escalation chains. Unlike prior work that provides harm taxonomies, harmful-task completion tests, execution-level safety benchmarks, or severity-aware simulation, our contribution is a reusable, trace-grounded severity instrument applied to the actual actions recorded in existing red-team logs. All code, prompts, and per-episode logs are released.