You Cannot Pick a Provider From the Price List: Market-Aware Routing for Open-Weight LLM Inference

Source facts

The preprint argues that open-weight inference routing must choose both a model and the provider serving it. Across live endpoints, models, task types, and three measurement waves, it reports that the same model can vary in quality, latency, availability, and price across providers. The proposed FACET router certifies provider-by-task feasibility and fails safe to an anchor before serving uncertified options; the abstract reports traffic moving from a premium anchor to cheaper certified endpoints in live runs.

Analysis

The paper treats inference providers as a dynamic market rather than interchangeable price points. For enterprise deployments, the operational primitive becomes measured, task-specific provider health and quality certification.

Uncertainty/Doubt

• This is an arXiv preprint. The live endpoint sample, measurement waves, evaluator bias, and savings claims were not independently audited here.