Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
3 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on cost-aware evaluation
3 notes
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The authors propose a new approach for evaluating security agents, focusing on cost-effectiveness rather than just success rate. They evaluate language models on offensive and defensive tasks, comparing performance at…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The authors propose a new approach for evaluating security agents by focusing on their cost-effectiveness instead of just their success rate in offensive and defensive tasks. They demonstrate this using language models…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The paper suggests that traditional methods for evaluating security agents, focusing on their offensive capabilities under generous inference budgets, are insufficient as they don't account for the cost of each…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
