Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs

Source facts

Across 18 base-instruct model pairs spanning Qwen, Llama, and Gemma, the authors report that harmfulness is more readily decoded from attention updates while MLP updates contribute more strongly to harm-selective refusal writing. They report that post-training strengthens refusal writing without systematically improving harmfulness decodability. Causal interventions and the proposed Sentinel routing system are evaluated across reasoning-enabled and other models.

Analysis

The paper separates harmfulness recognition from refusal generation, giving jailbreak and over-refusal audits two distinct internal targets. That decomposition may support more targeted safety interventions than treating refusal as a single undifferentiated behavior.

Uncertainty/Doubt

• This is an arXiv preprint. The model-pair selection, causal interventions, Sentinel results, and mechanistic conclusions were not independently reproduced here.