Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Two Axes of LLM Abstention: Answer Correctness and Question Answerability

Source

arXiv

arxiv.org/abs/2607.08456v1

Read the source

Source

Plain-Language Brief

The paper discusses the issue of language models providing incorrect or unanswerable responses. The author proposes two separate axes for abstention: one for answers the model would get wrong, and another for questions the model should not answer at all due to being unanswerable or based on false premises. The research finds that a single confidence score used by language models cannot distinguish between these cases. The paper suggests a solution to improve the model's ability to verify the premises and propose a policy that ensures both answerability and correctness.

What the Document Says

  • The author proposes two axes of abstention for language models: answer correctness and question answerability.
  • Five instruction-tuned models from three families (2B to 14B) were analyzed, with the finding that they are separate axes.
  • An ordinary answer-confidence score tells whether an answer is right but is nearly blind to whether the question is answerable.
  • A linear probe on hidden states is useful for determining question answerability, but not for assessing answer correctness.
  • The blind spot does not shrink with scale and is worst on naturally occurring false-premise questions (CREPE).
  • Answer-confidence, P(IK), P(True), and even asking the model outright whether a premise is false all stay near chance on CREPE.
  • A hidden-state probe reaches 0.69 to 0.77 AUROC on CREPE, indicating that the model represents a problem it will not report.
  • Instructing a model to check premises backfires because it then disputes sound and false premises alike.
  • The proposed solution involves routing the same instruction with the probe, which roughly triples challenge precision.
  • The policy turns the two axes into a calibrated policy that answers only when an answerability score and a correctness score each clear a separately certifies behavior.
  • The unanswerable-answer rate is controllable at every scale.
  • The wrong-answer rate is capped by model accuracy, so the guarantee tightens as threshold policy certifies both budgets at 0.75 coverage of correct answers.
  • At 14B, it is the only policy that certifies at all.

Key Concepts Mentioned

  • language models
  • abstention axes
  • answer correctness
  • question answerability
  • hidden states
  • premise checking
  • policy calibration

Claims Made by the Document

  • Language models should abstain from providing incorrect answers and unanswerable responses.
  • A single confidence score is not sufficient to distinguish between incorrect answers and questions that cannot be answered.
  • There are two separate axes of abstention: answer correctness and question answerability.
  • A linear probe on hidden states can determine question answerability but not answer correctness.

Evidence Used

  • Across five instruction-tuned models from three families (2B to 14B), the author finds that they are separate axes.
  • The analysis shows that answer-confidence is nearly blind to whether the question is answerable.
  • A hidden-state probe reaches a higher AUROC on CREPE than answer-confidence, indicating better performance.

Methods / Approach

  • analysis of instruction-tuned language models
  • linear probe on hidden states

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise. The usual recipe thresholds a single confidence score, which cannot tell these apart. Across five instruction-tuned models from three families (2B to 14B), we find they are separate axes. Ordinary answer-confidence tracks whether an answer is right but is nearly blind to whether the question is answerable; a linear probe on hidden states does the reverse. The blind spot does not shrink with scale. It is worst on naturally occurring false-premise questions (CREPE). There, answer-confidence, P(IK), P(True), and even asking the model outright whether a premise is false all stay near chance, while a hidden-state probe reaches 0.69 to 0.77 AUROC: the model represents a problem it will not report. This turns out to be fixable. Instructing a model to check premises backfires, because it then disputes sound and false premises alike (57% false challenges), unable to tell them apart; routing the same instruction with the probe roughly triples challenge precision. We turn the two axes into a calibrated policy that answers only when an answerability score and a correctness score each clear a separately certifies behave differently: the unanswerable-answer rate is controllable at every scale, while the wrong-answer rate is capped by model accuracy, so the guarantee tightens as threshold policy certifies both budgets at 0.75 coverage of correct answers, against 0.31 for a single threshold; at 14B it is the only policy that certifies at all.