HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention

Source facts

The preprint introduces HERA, a framework that constructs verifiable feasible/infeasible task pairs through controlled environment mutations and adapts the agent harness toward prior weaknesses. The abstract reports held-out abstention accuracy rising from 61.7% to 83.3% and feasible-task completion from 68.3% to 76.7%. It also reports transfer across 19 other LLMs and an estimated 85% lower cost for smaller models to match stronger-model performance.

Analysis

HERA treats abstention as a property of the harness and task environment, not only a model capability. The design makes infeasible-task generation and transfer evaluation explicit parts of reliability engineering.

Uncertainty/Doubt

• This is an arXiv preprint. The reported gains, transfer behavior, and cost estimate were not independently reproduced here.