FIELD NOTES TOPIC
ai auditing
3 notes, newest first.
The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images
Researchers audit the effectiveness of using visual tools like crop-and-zoom in multimodal language models by creating a causal graph to separate observation-mediated paths from action-induced shortcuts. They find that…
One Human, $N$ Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence
The paper discusses a scenario where a single human auditor has to evaluate multiple LLM (Language Learning Models) agents under a limited budget. The authors model this process as a noisy inspection over a two-level…
The Eticas AI Risk Taxonomy: Open Infrastructure for Operationalizing AI Audits
The Eticas AI Risk Taxonomy v2.0.0 is an open-source framework for operationalizing AI audits. It provides a standardized method for evaluating AI systems across various high-stakes domains, focusing on turning risks…
