From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Zhijun Gao, Jing Chen
• Venue: arXiv: cs.SE
• Published: 2026-08-20
• Collected: 2026-08-21
• URL: http://arxiv.org/abs/2608.20195v1
• PDF: https://arxiv.org/pdf/2608.20195v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

A study investigates how autonomous coding agents interact with technical documentation, revealing that they mostly work with internal documents and notes, and less often with classical technical documentation or API references. The link between consulting documentation and code editing is unclear, and there's no observed explicit validation sequence. Documentation trails code changes, and consultation is more self-initiated than failure-driven.

What the Document Says

• The study examines the behavior of autonomous coding agents in relation to technical documentation using two datasets: SWE-chat and AIDev.
• In the SWE-chat dataset, 557 agentic coding sessions resulted in 94,813 development events, including 3,033 documentation interactions.
• The AIDev dataset includes 33,097 agentic pull requests with 690,260 classified file-level change records.
• Four findings were made, challenging current documentation practices.
• First finding: Agents mainly interact with instruction files and working notes (60.5% of all interactions), rather than classical technical documentation or API references.
• Second finding: The link between consulting documentation and code editing is unclear; however, a stage-adjusted model places it above unity.
• Third finding: No explicit documentation-based validation sequence was observed, and consultation is associated with less immediate testing.
• Fourth finding: Consultation is more self-initiated (70.2%) than failure-driven (7.5%), and documentation trails code changes.
• The study shows that the assumptions about 'agent-friendly' documentation - actionability and verifiability - lack consistent behavioral support.
• The authors release their pipeline, coding scheme, and event-level data.

Key Concepts Mentioned

• autonomous coding agents
• technical documentation
• SWE-chat
• AIDev
• documentation interactions
• classical technical documentation
• API references
• actionability
• verifiability

Claims Made by the Document

• Agents mainly interact with instruction files and working notes, not classical technical documentation or API references.
• The link between consulting documentation and code editing is unclear but shows some support in a stage-adjusted model.
• No explicit documentation-based validation sequence was observed.
• Consultation is more self-initiated than failure-driven.
• Documentation trails code changes.

Evidence Used

• Data from SWE-chat and AIDev datasets.
• Analysis of 94,813 development events and 690,260 classified file-level change records.

Methods / Approach

• Behavior-grounded study
• Data analysis using two public datasets

Limitations Stated by the Authors

• The study only considers the behavior of autonomous coding agents in relation to technical documentation, not other factors that may influence their work.

Limitations Noted by the Agent

Agent interpretation — not from the source.

• The agent's ability to summarize findings and interpret results might be limited due to being based on provided text.

Open Questions

• Why is the link between consulting documentation and code editing unclear?
• What factors contribute to the observed behavior of autonomous coding agents in relation to technical documentation?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The study provides insights into how autonomous coding agents interact with technical documentation, but it does not explain why they behave in this manner or what other factors may influence their work.

Raw Abstract / Excerpt

Technical documentation is written for human developers, but an increasing share of software changes is now authored by autonomous coding agents. Which documents they consult, when, and what follows remain unknown. We conduct a behaviour-grounded study of agent-documentation interaction across two public datasets: 557 agentic coding sessions from SWE-chat, yielding 94,813 development events including 3,033 documentation interactions; and 33,097 agentic pull requests from AIDev, with 690,260 classified file-level change records. Four findings challenge current documentation practice. First, agents' documentation work is dominated by agent-facing artefacts: instruction files and working notes account for 60.5% of all documentation interactions, versus 10.6% for classical technical documentation and 1.3% for API references. Second, the link between consultation and code editing is unresolved: the adjacent transition probability is 0.002 and the unadjusted three-event lift 1.05, whereas a stage-adjusted model places it above unity (OR 1.33 [1.09, 1.62]); documentation creation is elevated unadjusted (lift 1.67) but its adjusted interval includes unity. Third, no explicit documentation-based validation sequence was observed, and consultation is associated with less immediate testing (lift 0.23, cluster CI 0.08-0.45; adjusted OR 0.39 [0.25, 0.60]). Fourth, consultation is self-initiated (70.2%) far more often than failure-driven (7.5%), and documentation trails code: among multi-commit pull requests changing both, code is touched first 4.7x more often. From these traces we derive a descriptive model of agent-documentation interaction as a two-lobed cycle rather than a linear journey, and show that two widely assumed properties of 'agent-friendly' documentation - actionability and verifiability - lack consistent behavioural support. We release our pipeline, coding scheme, and event-level data.