JEV-as-a-Judge: Accept When Confident, Escalate When Unsure

Source facts

The preprint evaluates a decision-only judge as an economical first pass and escalates low-confidence cases to stronger evaluation. It compares the judge with 16 generative and reward-model judges using blinded human adjudication and reports that a frozen confidence cascade retains most comparator accuracy at lower cost.

Analysis

The core pattern is selective evaluation: cheap automated acceptance for high-confidence cases, stronger review for uncertainty. This can make evaluator systems more affordable, but only if confidence is calibrated on the target distribution and the escalation policy catches difficult failure modes.

Uncertainty/Doubt

• This is an arXiv preprint; cost, accuracy, and calibration claims were not independently reproduced here.