Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

TraceLab: Characterizing Coding Agent Workloads for LLM Serving

Source

arXiv

arxiv.org/abs/2606.30560v1

Read the source

Source

Plain-Language Brief

Researchers collect and release a dataset of approximately 4,300 coding-agent sessions from their own use of Claude Code and Codex. The data shows that coding-agent workloads feature long autonomous loops, long contexts with short outputs, diverse and heavily-tailed tool calls, and high but imperfect prefix cache hit rates. These findings suggest opportunities for optimizing serving, such as lower-overhead tool calling, append-length-aware prefill, semantic-aware tool-latency prediction, and improved KV-cache management around human-paced gaps.

What the Document Says

  • The researchers collected a dataset of about 4,300 coding-agent sessions from their use of Claude Code and Codex.
  • The dataset contains approximately 350,000 LLM steps and 430,000 tool calls.
  • The analysis of the data shows that coding-agent workloads have long autonomous loops, long contexts with short outputs, diverse and heavily-tailed tool calls, and high but imperfect prefix cache hit rates.
  • The findings suggest opportunities for optimizing serving, including lower-overhead tool calling, append-length-aware prefill, semantic-aware tool-latency prediction, and improved KV-cache management around human-paced gaps.

Key Concepts Mentioned

  • coding agents
  • LLM
  • serving efficiency
  • workload patterns
  • public traces
  • benchmarks
  • Claude Code
  • Codex
  • long autonomous loops
  • long contexts
  • short outputs
  • tool calls
  • prefix cache hit rates
  • optimizing serving

Claims Made by the Document

  • Coding-agent workloads feature long autonomous loops, long contexts with short outputs, diverse and heavily-tailed tool calls, and high but imperfect prefix cache hit rates.

Evidence Used

  • The researchers analyzed a dataset of about 4,300 coding-agent sessions from their use of Claude Code and Codex.

Methods / Approach

  • Data collection and analysis of coding-agent sessions

Limitations Stated by the Authors

  • The data used for the analysis is limited to their own day-to-day use of Claude Code and Codex.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The analysis may not fully capture the workload patterns of coding agents across all applications or model families.
  • The findings may not be universally applicable, as they are based on a specific dataset.

Open Questions

  • How do other coding-agent applications and model families compare in terms of workload patterns?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The research focuses on understanding real workload patterns for coding agents using Claude Code and Codex. It could be interesting to investigate how these findings apply to other coding-agent applications and model families.
  • This study provides insights into optimizing serving for coding agents, but it's essential to consider the limitations mentioned in the text when interpreting and applying the findings.

Raw Abstract / Excerpt

Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging. Progress on this challenge requires understanding real workload patterns, yet the data needed for such analysis is largely absent. Existing public traces and benchmarks do not capture real, day-to-day coding-agent usage across multiple agents and model families for serving-system analysis. To help fill this gap, we collect and release a trace of roughly 4,300 coding-agent sessions, containing about 350,000 LLM steps and 430,000 tool calls from our own day-to-day use of Claude Code and Codex. Our analysis shows that coding-agent workloads feature long autonomous loops, long contexts with short outputs, diverse and heavily-tailed tool calls, and high but imperfect prefix cache hit rates. These findings point to concrete opportunities for optimizing serving, including lower-overhead tool calling, append-length-aware prefill, semantic-aware tool-latency prediction, and improved KV-cache management around human-paced gaps. We release the dataset, trace collection pipeline, and analysis code at https://github.com/uw-syfi/TraceLab.git; the project website is https://tracelab.cs.washington.edu.