Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Scrouting: Cost-Aware Routing of Coding Agents by Scouting the Repository First

Source

arXiv

arxiv.org/abs/2608.04804v1

Read the source

Source

Plain-Language Brief

The researchers introduce SuperScout, a cost-effective solution for routing language models to resolve software issues in repositories. SuperScout first explores the repository and produces a structured handoff, which is then used by a resume-based router to dispatch tasks to four frontier fixers. The handoff seems to redistribute solving ability among the fixers without adding new abilities, improving cost efficiency.

What the Document Says

  • The paper presents SuperScout, a system that routes language models for resolving repository-level software issues.
  • SuperScout starts by exploring the repository with a 7B searcher (SuperScout-7B) and produces a structured handoff.
  • The sandbox-verified reproduction claims are checked, and false claims are removed before delivery.
  • A resume-based router dispatches the task to one of four frontier fixers based on hidden states and the task text.
  • Adding a new fixer does not require retraining.
  • On SWE-bench Pro's full Python slice under the capped budget tier, SuperScout matches the best model's solve rate at about a fifth of the total cost per solve.
  • The no-router ablation (always the cheapest fixer with the handoff) ties the routed system on this benchmark.
  • A paired calibration study suggests that the handoff redistributes solving ability, lifting the cheaper fixers and slightly hurting the strongest.
  • The searcher's compute adds less than half a cent of GPU time per task.

Key Concepts Mentioned

  • SuperScout
  • repository-level software issues
  • language models
  • searcher
  • handoff
  • resume-based router
  • frontier fixers
  • SWE-bench Pro

Claims Made by the Document

  • SuperScout matches the best model's solve rate at about a fifth of the total cost per solve.
  • The handoff redistributes solving ability, lifting cheaper fixers and slightly hurting the strongest.

Evidence Used

  • Results from SWE-bench Pro's full Python slice under the capped budget tier.
  • A paired calibration study

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Frontier language models can resolve repository-level software issues, but each attempt is expensive, and existing routers select a model from the issue text alone. We present SuperScout, which routes after scouting the repository: a 7B searcher, SuperScout-7B, first explores the repository and produces a structured handoff whose reproduction claims are sandbox-verified, with false claims stripped before delivery. The searcher's hidden states, together with the task text, then feed a resume-based router that dispatches the task to one of four frontier fixers. Adding a new fixer requires no retraining. On the full Python slice of SWE-bench Pro (266 tasks) under the benchmark's official capped budget tier, SuperScout matches the best single model's solve rate (159 of 266 for SuperScout, 158 for the best model) at about a fifth of the total cost per solve, and the reported configuration sits above the random traffic-splitting baseline. A no-router ablation, always the cheapest fixer with the handoff, ties the routed system on this benchmark, so the handoff rather than the routing decision carries the result. A paired calibration study points to the mechanism: the handoff appears to redistribute rather than add solving ability, lifting the three cheaper fixers while slightly hurting the strongest, though at $N=99$ the per-fixer effects are directional only; the searcher's hidden states improve cost routing on the calibration labels while the handoff's own text does not. The searcher's compute adds less than half a cent of GPU time per task.