Steerability via constraints: a substrate for scalable oversight of coding agents

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Thomas Winninger
• Venue: arXiv: cs.AI
• Published: 2026-07-02
• Collected: 2026-07-03
• URL: http://arxiv.org/abs/2607.02389v1
• PDF: https://arxiv.org/pdf/2607.02389v1
• DOI: UNKNOWN

Plain-Language Brief

The author proposes a method for managing coding agents that mimics traditional methods used in large engineering teams, such as access control and strict coding conventions enforced by tools. They present a controlled experiment where a reviewer named Gemma 4 e4b inspects a Python codebase containing backdoors, demonstrating improved recall rates with the use of this method.

What the Document Says

• Coding agents are capable but human oversight is a bottleneck.
• Unconstrained coding agents present security risks and make human review costly.
• The author suggests transferring traditional management methods for large engineering teams to coding agents, including access control, network policies, and strict coding conventions enforced by tooling.
• These methods are claimed to be cheaper than recent agentic scaffolding.
• The author sketches a system based on this principle.
• A controlled experiment is reported where a reviewer named Gemma 4 e4b inspects a Python codebase containing 11 inserted backdoors.
• Recall increases from 54.5% to 90.9% with the use of constrained substrate and tools, with each contributing independently.
• The principles are said to extend to languages like Rust.

Key Concepts Mentioned

• coding agents
• human oversight
• security risks
• codebase scalability
• access control
• network policies
• strict coding conventions
• tooling
• Gemma 4 e4b
• Python
• Rust
• backdoors
• recall

Claims Made by the Document

• Traditional management methods for large engineering teams can be transferred to coding agents.
• These methods are cheaper than recent agentic scaffolding.
• The use of constrained substrate and tools improves recall rates.

Evidence Used

• The author's proposed system and controlled experiment

Methods / Approach

• Controlled experiment
• Proposal of a system based on traditional management methods for large engineering teams

Limitations Stated by the Authors

The author does not provide any specific limitations in the text.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The agent does not have information about the scope and rigor of the controlled experiment, or the specifics of how traditional management methods are implemented for coding agents.

Open Questions

• What is the scope and rigor of the controlled experiment?
• How are traditional management methods specifically implemented for coding agents?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The text does not provide detailed information about the experimental setup or the specifics of how the traditional management methods are applied to coding agents.

Raw Abstract / Excerpt

Coding agents are capable; human oversight is the bottleneck. Unconstrained agents introduce security risks, erode codebase scalability, and make human review increasingly costly. We argue that the same methods used for decades to manage large human engineering teams: access control, network policies, strict coding conventions enforced by tooling; transfer directly to coding agents, and are cheaper (in token) than recent agentic scaffolding. We sketch a start-to-end system on this principle, and report a controlled experiment in scalable oversight: a small reviewer (Gemma 4 e4b) inspects a Python codebase containing 11 inserted backdoors. Recall rises from 54.5% (unconstrained, no tools) to 90.9% (constrained substrate plus a ~200-LoC docs CLI), with substrate and tools contributing independently. We choose Python deliberately: substrate-level oversight gains are largest where the language gives the fewest guarantees by default; the principles extend to languages like Rust.