Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Adam Fisch, Shubhendu Trivedi, Fantine Huot, William W. Cohen, Michael Kaisers, Mirella Lapata, Kate Larson, Jacob Eisenstein
• Venue: arXiv: cs.AI
• Published: 2026-08-20
• Collected: 2026-08-21
• URL: http://arxiv.org/abs/2608.20316v1
• PDF: https://arxiv.org/pdf/2608.20316v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

Researchers have developed a system that routes AI queries to the most effective and efficient model based on costly value estimation. The system, called Pandora's Router, uses a trade-off between fast but noisy estimators and accurate but expensive ones to decide when to refine the value estimate. Experiments show that the centralized policy matches the routing quality of exhaustive estimation while querying the expensive estimator less often. In a decentralized setting, the system improves allocative efficiency with accurate estimates, but can increase strategic specialist's utility at the expense of others when competing estimates are noisy.

What the Document Says

• The research focuses on heterogeneous AI systems and proposes a method to route queries efficiently and cost-effectively.
• Two types of value estimators are mentioned: cheap but noisy ones (e.g., embedding-based predictors) and accurate but expensive ones (e.g., fine-tuned models).
• The researchers formulate the tradeoff between cost and accuracy as an instance of Pandora's Box problem.
• They introduce a centralized policy called Pandora's Router that determines whether to refine the value estimate based on closed-form value-of-information expressions.
• An extension to a decentralized setting is also proposed, named Pandora's Bidder.
• Experiments across three domains show that Pandora's Router achieves routing quality comparable to exhaustive estimation while querying the expensive estimator less frequently.
• In the decentralized setting, the system improves allocative efficiency with accurate estimates but can increase strategic specialist's utility at the expense of others when competing estimates are noisy.

Key Concepts Mentioned

• Heterogeneous AI systems
• Value estimation
• Costly inspection (Pandora's Box problem)
• Centralized policy (Pandora's Router)
• Decentralized setting (Pandora's Bidder)
• Allocative efficiency

Claims Made by the Document

• Heterogeneous AI systems can improve quality and efficiency by routing queries to the specialist who can answer most effectively at the lowest cost.
• The centralized policy (Pandora's Router) matches the routing quality of exhaustive estimation while querying the expensive estimator less often.
• In the decentralized setting, value-of-information reasoning improves allocative efficiency when competing estimates are accurate.

Evidence Used

• The text describes the methods used in the research and presents experimental results across three domains.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing queries to the specialist who can answer most effectively at the lowest cost. Routing requires estimating each specialist's expected return, but this value estimation has a cost. Cheap estimators (e.g., embedding-based predictors) are fast but noisy, while accurate estimators (e.g., fine-tuned models with access to retrieval results or partial reasoning traces) are expensive. We formalize this tradeoff as an instance of Pandora's Box, the classical problem of optimal search with costly inspection. Under a Gaussian signal model, the resulting policies have closed-form value-of-information expressions that determine, for each specialist and input, whether refining the value estimate is worth its cost. We call the centralized policy Pandora's Router. We extend this to a decentralized setting, Pandora's Bidder, where specialists independently decide whether to invest in self-assessment before accepting an offered price to claim a query. Experiments across three domains---a standard multi-LLM benchmark, retrieval-augmented specialists, and LLMs with variable inference-time reasoning---show that Pandora's Router matches the routing quality of exhaustive estimation, while querying the expensive estimator far less often. In the decentralized setting, value-of-information reasoning improves allocative efficiency when competing estimates are accurate; when competing estimates are noisy, however, it can increase the strategic specialist's utility at the expense of others.