SAFARI: Scaling Long Horizon Agentic Fault Attribution via Active Investigation

Source

Source name: arXiv
Source kind: paper
Authors / organization: Chenyang Zhu, Jiayu Yao, Kushal Chawla, Youbing Yin, Nathan Wolfe, Pengshan Cai, Jingyu Wu, Spencer Hong, Sangwoo Cho, Shi-Xiong Zhang, Daben Liu, Sambit Sahu, Erin Babinsky
Venue: arXiv: cs.AI
Published: 2026-06-23
Collected: 2026-06-24
URL: http://arxiv.org/abs/2606.24626v1
PDF: https://arxiv.org/pdf/2606.24626v1
DOI: UNKNOWN

Plain-Language Brief

Researchers have developed SAFARI, a framework that helps diagnose failures in autonomous agents by using a specialized toolbox for reading and searching trajectory segments and a persistent Short-Term Memory (STM) for cross-turn reasoning. This allows the system to maintain diagnostic accuracy even when agentic traces exceed context limits, as compared to state-of-the-art results.

What the Document Says

Researchers developed SAFARI: Scaling long-horizon Agentic Fault Attribution via active Investigation.
SAFARI replaces linear context loading with a tool-augmented diagnostic loop.
SAFARI equips LLMs (Large Language Models) with a specialized toolbox for reading and searching trajectory segments.
SAFARI also uses a persistent Short-Term Memory (STM) for cross-turn reasoning.
SAFARI experiments show it outperforms state-of-the-art results by 20% on the Who&When dataset within a 1M token budget, and by 19% on TRAIL GAIA subset on a 25K token budget.
Even when the target fault resides 5x beyond the model's native context window, SAFARI maintains a precision of 0.58 while traditional evaluators fail entirely.

Key Concepts Mentioned

SAFARI
autonomous agents
fault diagnosis
large language models (LLMs)
Short-Term Memory (STM)
Who&When dataset
TRAIL GAIA subset
context limits
attention dilution

Claims Made by the Document

SAFARI outperforms state-of-the-art results.
SAFARI maintains a higher precision than traditional evaluators when the target fault is beyond the model's native context window.

Evidence Used

Experiments demonstrate that SAFARI outperforms state-of-the-art results by 20% and 19%, respectively, on two different datasets.
SAFARI maintains a precision of 0.58 when the target fault resides beyond the model's native context window while traditional evaluators fail entirely.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

As autonomous agents tackle increasingly complex multi-step, multi-agent tasks, their execution trajectories have scaled beyond the constraints of even the largest context windows. Current methods for effectively diagnosing agent failures load the full trajectory into an LLM's context window, which suffers from attention dilution and fails when agentic traces inevitably exceed context limits. To address this, we introduce SAFARI (Scaling long-horizon Agentic Fault AttRibution via active Investigation), a framework that replaces linear context loading with a tool-augmented diagnostic loop. By equipping LLMs with a specialized toolbox to read and search trajectory segments alongside a persistent Short-Term Memory (STM) for cross-turn reasoning, SAFARI effectively decouples diagnostic accuracy from architectural context limits. Our experiments demonstrate that SAFARI outperforms state-of-the-art results by 20% on the Who&When dataset within a 1M token budget, and by 19% on TRAIL GAIA subset on a 25K token budget. Most significantly, SAFARI maintains a 0.58 precision even when the target fault resides 5x beyond the model's native context window, a scenario where traditional evaluators fail entirely.