Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs

Source

arXiv

arxiv.org/abs/2607.28573v1

Read the source

Source

Plain-Language Brief

This research study explores the effectiveness of inference-time scaling for computer-use agents (CUAs) that run locally on devices. The authors investigate various dimensions such as context, time, structure, and parallelism to understand how additional computation during execution affects these local models' performance under strict hardware constraints. The study finds that while contextual and temporal scaling can improve certain aspects like trajectory stability and reducing max-step stalls, they may lead to diminishing returns or different types of failures as token costs increase. Structural decomposition and parallel scaling also present unique challenges and benefits for local two-stage agents. The researchers conclude that efficient local CUAs need selective compute allocation, failure-aware control mechanisms, and agentic frameworks designed considering the capabilities and limitations of these devices.

What the Document Says

  • The study investigates inference-time scaling for computer-use agents (CUAs) running locally on devices.
  • They examine contextual, temporal, structural, and parallel dimensions.
  • Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B models were evaluated on the OSWorld benchmark.
  • Contextual scaling provides historical grounding that improves trajectory stability and task accuracy but saturates as token cost increases.
  • Temporal scaling reduces max-step stalls without substantially improving task success.
  • Structural decomposition can introduce planning and formatting overhead in local two-stage agents.
  • Parallel scaling partially mitigates failures at a substantial computational cost.
  • The findings suggest that efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks.

Key Concepts Mentioned

  • computer-use agents
  • inference-time scaling
  • local models
  • contextual scaling
  • temporal scaling
  • structural decomposition
  • parallel scaling

Claims Made by the Document

  • Inference-time scaling can improve local CUAs.
  • Contextual scaling improves trajectory stability and task accuracy.
  • Temporal scaling reduces max-step stalls without improving task success substantially.
  • Structural decomposition can introduce planning and formatting overhead.
  • Parallel scaling partially mitigates failures at a substantial computational cost.
  • Efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks.

Evidence Used

  • Results from the evaluation of Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on the OSWorld benchmark.
  • Observations about failures shifting from repetitive or stalled trajectories toward premature false successes.

Methods / Approach

  • Empirical study of inference-time scaling in local CUAs across different dimensions.

Limitations Stated by the Authors

  • The study focuses on three specific models and the OSWorld benchmark.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The text lacks information about the criteria used to select these specific models.
  • It is unclear whether the results are generalizable to other benchmarks or models.

Open Questions

  • What other factors influence the performance of local CUAs under strict hardware constraints?
  • Can alternative scaling methods be more effective in improving the performance of local CUAs?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The study aims to understand the effectiveness of inference-time scaling for resource-constrained local models.
  • Contextual and temporal scaling have their own benefits and limitations, with contextual scaling being effective up to a point but leading to different types of failures as token costs increase.

Raw Abstract / Excerpt

Deploying autonomous computer-use agents (CUAs) locally is increasingly important for privacy, cost efficiency, and practical usability, yet improving their performance under strict hardware constraints remains challenging. While recent studies show that inference-time scaling can improve frontier computer-use agents through additional computation during execution, its effectiveness for resource-constrained local models remains poorly understood. We present a systematic empirical study of inference-time scaling in local CUAs across contextual, temporal, structural, and parallel dimensions. We evaluate Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on the OSWorld benchmark. Our results show that additional computation often yields diminishing returns while changing failure modes. Contextual scaling provides historical grounding that improves trajectory stability and task accuracy, but its gains saturate as token cost increases and failures shift from repetitive or stalled trajectories toward premature false successes. Temporal scaling similarly reduces max-step stalls, yet does not substantially improve task success, indicating that longer horizons often extend erroneous trajectories rather than correct them. We further find that structural decomposition can introduce planning and formatting overhead in local two-stage agents, while parallel scaling partially mitigates these failures at a substantial computational cost. Overall, our findings suggest that efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks designed around the capabilities and limitations of local models.