Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs

Source

arXiv

arxiv.org/abs/2607.28573v1

Read the source

Source

Plain-Language Brief

The paper presents an empirical study on the effectiveness of inference-time scaling in locally deployed autonomous computer-use agents (CUAs) under strict hardware constraints. The research focuses on four dimensions: contextual, temporal, structural, and parallel. The authors find that while additional computation can provide some benefits, there are diminishing returns and a shift in failure modes as token cost increases. Contextual scaling helps improve task accuracy by providing historical grounding but saturates at high costs, while temporal scaling reduces max-step stalls but does not significantly improve task success. Structural decomposition can introduce planning and formatting overhead, while parallel scaling mitigates these failures at a higher computational cost.

What the Document Says

  • The paper is about studying the impact of inference-time scaling on locally deployed autonomous computer-use agents (CUAs).
  • It explores four dimensions: contextual, temporal, structural, and parallel.
  • Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B were evaluated on the OSWorld benchmark.
  • Additional computation often provides diminishing returns with a shift in failure modes as token cost increases.
  • Contextual scaling improves trajectory stability and task accuracy but saturates at high costs and leads to premature false successes.
  • Temporal scaling reduces max-step stalls but does not substantially improve task success, suggesting that longer horizons extend erroneous trajectories rather than correct them.
  • Structural decomposition can introduce planning and formatting overhead in local two-stage agents.
  • Parallel scaling partially mitigates these failures at a substantial computational cost.

Key Concepts Mentioned

  • inference-time scaling
  • locally deployed autonomous computer-use agents (CUAs)
  • hardware constraints
  • contextual scaling
  • temporal scaling
  • structural decomposition
  • parallel scaling
  • OSWorld benchmark

Claims Made by the Document

  • Additional computation often yields diminishing returns while changing failure modes.
  • Contextual scaling provides historical grounding that improves trajectory stability and task accuracy but saturates as token cost increases.
  • Temporal scaling reduces max-step stalls, yet does not substantially improve task success.
  • Structural decomposition can introduce planning and formatting overhead in local two-stage agents.
  • Parallel scaling partially mitigates failures at a substantial computational cost.

Evidence Used

  • Results from evaluating Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on the OSWorld benchmark

Methods / Approach

  • Empirical study
  • Evaluation on the OSWorld benchmark

Limitations Stated by the Authors

  • The paper does not discuss potential solutions for the identified limitations.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The agent might miss any potential insights from the authors' interpretations or discussions beyond their empirical findings.

Open Questions

  • What are possible solutions to mitigate failures while maintaining efficiency in local CUAs?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The paper provides insights into the effectiveness of inference-time scaling for resource-constrained local models, but it does not offer immediate solutions for addressing the identified challenges.

Raw Abstract / Excerpt

Deploying autonomous computer-use agents (CUAs) locally is increasingly important for privacy, cost efficiency, and practical usability, yet improving their performance under strict hardware constraints remains challenging. While recent studies show that inference-time scaling can improve frontier computer-use agents through additional computation during execution, its effectiveness for resource-constrained local models remains poorly understood. We present a systematic empirical study of inference-time scaling in local CUAs across contextual, temporal, structural, and parallel dimensions. We evaluate Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on the OSWorld benchmark. Our results show that additional computation often yields diminishing returns while changing failure modes. Contextual scaling provides historical grounding that improves trajectory stability and task accuracy, but its gains saturate as token cost increases and failures shift from repetitive or stalled trajectories toward premature false successes. Temporal scaling similarly reduces max-step stalls, yet does not substantially improve task success, indicating that longer horizons often extend erroneous trajectories rather than correct them. We further find that structural decomposition can introduce planning and formatting overhead in local two-stage agents, while parallel scaling partially mitigates these failures at a substantial computational cost. Overall, our findings suggest that efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks designed around the capabilities and limitations of local models.