Learning the Cost of Reliable Inference

Source facts

The paper proposes a procurement platform where model providers compete on task-level token prices through a reverse second-price auction. The platform learns provider quality and routes requests to a cost-competitive provider subject to a quality threshold. Experiments with Llama and Qwen providers report that the margin of the most cost-competitive provider varies from 10% to 71%.

Analysis

The contribution is a market-design proposal for inference routing, not evidence that such a marketplace is deployed. It frames reliability-constrained model selection as a procurement problem and suggests that fixed-price inference markets may leave savings available to buyers with measurable quality requirements.

Uncertainty/Doubt

• This is an arXiv preprint. The auction assumptions, provider-quality estimates, model economics, and reported savings were not independently reproduced here; no operating marketplace was verified.