FIELD NOTES TOPIC
cost-aware evaluation
3 notes, newest first.
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The authors propose a new approach for evaluating security agents, focusing on cost-effectiveness rather than just success rate. They evaluate language models on offensive and defensive tasks, comparing performance at…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The authors propose a new approach for evaluating security agents by focusing on their cost-effectiveness instead of just their success rate in offensive and defensive tasks. They demonstrate this using language models…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The paper suggests that traditional methods for evaluating security agents, focusing on their offensive capabilities under generous inference budgets, are insufficient as they don't account for the cost of each…
