Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision

Source

arXiv

arxiv.org/abs/2606.32038v1

Read the source

Source

Plain-Language Brief

The study examines how language models (LMs) generate self-explanations for their predictions, and finds that when training LMs to explain which aspects of their inputs influenced their behavior, they often produce more faithful explanations than superficial imitation. This phenomenon is referred to as 'introspective coupling'. The study also finds that this introspective coupling occurs when training explanations remain correlated with current behaviors over time, and that it can track behavior shifts without requiring updated supervision.

What the Document Says

  • The researchers examine language models (LMs) generating self-explanations for their predictions.
  • They find that LMs trained on fixed counterfactual explanations derived from earlier checkpoints or similar models in different families produce more faithful explanations to their own behaviors than the training targets.
  • This phenomenon, called 'introspective coupling', happens when training explanations remain correlated with current behaviors over the course of training.
  • The study shows that introspective coupling can track behavior shifts when explanation training is provided concurrently with other post-training objectives.
  • The results suggest that fixed datasets of counterfactual explanations can provide scalable and generalizable post-training signal for introspection.
  • The phenomenon appears in multiple tasks, including sycophancy and refusal, and is robust to label noise.

Key Concepts Mentioned

  • language models
  • self-explanation training
  • counterfactual behavior
  • introspective coupling
  • behavioral change
  • post-training signal

Claims Made by the Document

  • LMs trained on fixed counterfactual explanations produce more faithful self-explanations.
  • Introspective coupling occurs when training explanations remain correlated with current behaviors over the course of training.
  • Introspective coupling can track behavior shifts when explanation training is provided concurrently with other post-training objectives.
  • Fixed datasets of counterfactual explanations can provide scalable and generalizable post-training signal for introspection.

Evidence Used

  • Study on LMs trained to explain which features influenced their behavior using counterfactual behavior on modified inputs as supervision.
  • Findings from the study showing that LMs frequently produce explanations more faithful to their own current behaviors than to those of their training targets.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

When does training language models (LMs) to generate explanations of their predictions yield faithful introspection, rather than superficial imitation? We study LMs trained to explain which features of their inputs influenced their behavior, using models' counterfactual behavior on modified inputs as supervision. Surprisingly, we find that LMs trained on fixed counterfactual explanations derived from earlier checkpoints of themselves, or even from behaviorally similar models in different families, frequently produce explanations more faithful to their own current behaviors than to those of their training targets. This 'introspective' coupling between LM explanations and behaviors occurs when training explanations remain sufficiently correlated with current behaviors over the course of training, even as behaviors themselves shift. We also show that introspective coupling tracks behavior shifts: when explanation training is provided concurrently with other post-training objectives, explanations track those shifts without requiring updated supervision. This phenomenon appears in multiple tasks, including sycophancy and refusal, and is robust to label noise. Overall, our results show that even fixed datasets of counterfactual explanations can provide scalable and generalizable post-training signal for introspection.