Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note4 min read

Two Axes of LLM Abstention: Answer Correctness and Question Answerability

Source

arXiv

arxiv.org/abs/2607.08456v1

Read the source

Source

Plain-Language Brief

A study finds that large language models often struggle with two types of errors: answering questions incorrectly and attempting to answer unanswerable or misleading questions. The researchers propose a method to separate these two issues, which can lead to more accurate and reliable responses.

What the Document Says

  • The study suggests that large language models should avoid giving wrong answers and refrain from answering certain questions that are unanswerable or based on false premises.
  • The authors note that a single confidence score used by these models cannot differentiate between correct and incorrect answers, nor can it determine if a question is answerable or not.
  • They find that two separate axes exist in the models: one focusing on answer correctness (answer-confidence) and another on question answerability (hidden-state probe).
  • The study reveals that the ability of the models to recognize unanswerable questions doesn't improve with scale, particularly on false-premise questions.
  • The researchers found that instructing the model to check premises leads to a high rate of incorrect challenges, as it disputes both sound and false premises equally.
  • They propose a method to combine these two axes into a calibrated policy that ensures the model answers only when both an answerability score and a correctness score meet certain thresholds.
  • The study states that this approach allows for controlling the unanswerable-answer rate at every scale, while capping the wrong-answer rate based on the model's accuracy.

Key Concepts Mentioned

  • large language models
  • answer correctness
  • question answerability
  • false premise questions

Claims Made by the Document

  • A single confidence score cannot differentiate between correct and incorrect answers, nor determine if a question is answerable or not.
  • Two separate axes exist in large language models: one focusing on answer correctness (answer-confidence) and another on question answerability (hidden-state probe).
  • The ability of the models to recognize unanswerable questions doesn't improve with scale, particularly on false-premise questions.
  • Instructing the model to check premises leads to a high rate of incorrect challenges.
  • Combining these two axes into a calibrated policy allows for controlling the unanswerable-answer rate and capping the wrong-answer rate based on the model's accuracy.

Evidence Used

  • The study compares five instruction-tuned models from three families (2B to 14B) to analyze their performance.
  • They use a linear probe on hidden states and observe the results.
  • They test the models on false-premise questions and measure the performance with AUROC.

Methods / Approach

  • comparing multiple instruction-tuned language models
  • using a linear probe on hidden states
  • testing the models on false-premise questions
  • measuring performance using AUROC

Limitations Stated by the Authors

  • The study does not provide specific details about the datasets or evaluation metrics used.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The study only focuses on a specific aspect of language models' behavior, and other issues may exist.
  • The effectiveness of the proposed method in real-world scenarios might be different from its performance in controlled conditions.

Open Questions

  • What are the datasets and evaluation metrics used in this study?
  • How effective is the proposed method in real-world scenarios?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The text mentions AUROC but does not define it. It stands for Area Under the Receiver Operating Characteristic Curve, a measure of the model's performance.

Raw Abstract / Excerpt

A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones resting on a false premise. The usual recipe thresholds a single confidence score, which cannot tell these apart. Across five instruction-tuned models from three families (2B to 14B), we find they are separate axes. Ordinary answer-confidence tracks whether an answer is right but is nearly blind to whether the question is answerable; a linear probe on hidden states does the reverse. The blind spot does not shrink with scale. It is worst on naturally occurring false-premise questions (CREPE). There, answer-confidence, P(IK), P(True), and even asking the model outright whether a premise is false all stay near chance, while a hidden-state probe reaches 0.69 to 0.77 AUROC: the model represents a problem it will not report. This turns out to be fixable. Instructing a model to check premises backfires, because it then disputes sound and false premises alike (57% false challenges), unable to tell them apart; routing the same instruction with the probe roughly triples challenge precision. We turn the two axes into a calibrated policy that answers only when an answerability score and a correctness score each clear a separately certifies behave differently: the unanswerable-answer rate is controllable at every scale, while the wrong-answer rate is capped by model accuracy, so the guarantee tightens as threshold policy certifies both budgets at 0.75 coverage of correct answers, against 0.31 for a single threshold; at 14B it is the only policy that certifies at all.