FIELD NOTES TOPIC
security evaluation
2 notes, newest first.
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The authors propose a new approach for evaluating security agents, focusing on cost-effectiveness rather than just success rate. They evaluate language models on offensive and defensive tasks, comparing performance at…
Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents
The paper proposes an action-graded harm rubric for evaluating AI agents' behavior during cybersecurity attacks. This scale assesses the severity of an agent's actions based on factors like reversibility, scope…
