Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories

Source

arXiv

arxiv.org/abs/2608.06346v1

Read the source

Source

  • Source name: arXiv
  • Source kind: paper
  • Authors / organization: Yunjia Qi, Zehua Yin, Xintong Shi, Hao Peng, Songyuanyi Lu, Yixian Liu, Richeng Xuan, Yuhong Liu, Zhichao Hu, Xiaozhi Wang, Lei Hou, Bin Xu, Juanzi Li
  • Venue: arXiv: cs.AI
  • Published: 2026-08-06
  • Collected: 2026-08-07
  • URL: http://arxiv.org/abs/2608.06346v1
  • PDF: https://arxiv.org/pdf/2608.06346v1
  • DOI: UNKNOWN

Plain-Language Brief

In this study, researchers present TrajDebug, a framework designed to identify critical errors in long trajectories of large language models (LLM) used in complex domains. The framework addresses two challenges: first, it simplifies the identification of individual errors across distant instructions, observations, and prior context; second, it identifies which local errors are responsible for the final failure. The researchers also created TrajErrBench, a benchmark with 486 annotated failed trajectories from existing datasets. Experiments showed that TrajDebug outperformed other baselines and provided useful feedback for improving agent success.

What the Document Says

  • The work presents TrajDebug, an error-lifecycle tracing framework for long trajectories in LLM-based agentic systems.
  • TrajDebug addresses the challenge of identifying individual errors by using multi-granularity history compression and evidence-based error identification.
  • It supports critical attribution by tracing each error's resolution status and terminal impact.
  • The researchers constructed TrajErrBench, a benchmark containing 486 annotated failed trajectories from Tau2Bench and SWE-Bench Pro.
  • Experiments show that TrajDebug outperforms existing baselines and provides actionable feedback for improving downstream agent success.
  • The codes and data will be released to facilitate further research.

Key Concepts Mentioned

  • LLM-based agentic systems
  • Trajectories
  • Critical error detection
  • Error lifecycle
  • Multi-granularity history compression
  • Evidence-based error identification
  • Critical attribution
  • Tau2Bench
  • SWE-Bench Pro
  • TrajErrBench

Claims Made by the Document

  • TrajDebug addresses challenges in critical error detection.
  • TrajDebug outperforms existing baselines and provides actionable feedback for improving downstream agent success.

Evidence Used

  • The text mentions that TrajDebug was proposed to address the difficulties in long-trajectory error discovery and critical attribution.
  • Experiments results show that TrajDebug outperformed other baselines according to the authors.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

  • The paper does not provide details about the methods used in TrajDebug.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The agent may struggle to understand specific technical details without additional research or context.

Open Questions

  • What are the specific methods used in TrajDebug for multi-granularity history compression and evidence-based error identification?
  • Were there any specific baselines used for comparison with TrajDebug?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The text does not provide detailed information about the methods used in TrajDebug. Further investigation may be needed to understand the approach.
  • While it's stated that experiments were conducted across diverse agent benchmarks, no specific baselines are mentioned for comparison with TrajDebug.

Raw Abstract / Excerpt

LLM-based agentic systems have shown remarkable capabilities in complex domains, while suffering from cascading errors and difficulty in debugging. Critical error detection aims to locate the earliest error step in a failed trajectory that is responsible for the final failure. However, progress faces two main challenges. First, long trajectories make it difficult to identify individual errors, since the evidence for judging a step may be scattered across distant instructions, observations, and prior context. Second, failed trajectories often contain multiple local errors with different downstream effects, only some of which remain responsible for the final failure. In this work, we propose TrajDebug, an error-lifecycle tracing framework that addresses long-trajectory error discovery with multi-granularity history compression and evidence-based error identification, and supports critical attribution by tracing each error's resolution status and terminal impact. We further construct TrajErrBench, a benchmark of 486 manually annotated failed trajectories from Tau2Bench and SWE-Bench Pro, covering realistic tool-use and coding scenarios. Experiments across diverse agent benchmarks show that TrajDebug achieves the best overall performance over existing baselines, and application studies further demonstrate that its diagnoses provide actionable feedback for improving downstream agent success. We will release the codes and data to facilitate further research.