Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

DRIFTLENS: Measuring Memory-Induced Reasoning Drift in Personalized Language Models

Source

arXiv

arxiv.org/abs/2607.02374v1

Read the source

Source

Plain-Language Brief

Researchers introduce DRIFTLENS, a framework to measure changes in the reasoning process used by personalized language models due to user attributes, preferences, and prior context. The study shows that these memory-induced reasoning drifts can occur even when final answers remain relevant and plausible.

What the Document Says

  • The paper discusses the effect of personalization on a model's reasoning trajectory for open-ended questions.
  • DRIFTLENS is introduced as a framework to quantify the divergence between a question's no-memory trajectory and its trajectory under injected user-attribute memory.
  • The study uses four LLMs (Language Learning Models) and 10 user-attribute categories such as age, occupation, and disability.
  • DRIFTLENS is able to distinguish content-free pragmatic noise from substantive reasoning changes.
  • User-attribute memory induces medium-to-large reasoning drift above each model's pragmatic-noise floor.
  • Effects on downstream capability, helpfulness, and instruction following are model-and reward-dependent.
  • GRPO- and DPO-based post-training methods are evaluated for reducing drift but neither uniformly dominates.

Key Concepts Mentioned

  • personalized language models
  • DRIFTLENS
  • reasoning drift
  • user attributes
  • LLMs
  • pragmatic noise

Claims Made by the Document

  • User-attribute memory induces medium-to-large reasoning drift above each model's pragmatic-noise floor.
  • GRPO- and DPO-based post-training methods reduce drift but neither uniformly dominates.

Evidence Used

  • Across four LLMs and 10 user-attribute categories, including age, occupation, and disability, user-attribute memory induces medium-to-large reasoning drift above each model's pragmatic-noise floor.
  • Both GRPO- and DPO-based post-training methods reduce drift but neither uniformly dominates.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Personalization changes what a model says to a user; we show that it can also change the reasoning trajectory used to justify the response. Modern LLMs personalize interactions by storing user attributes, preferences, and prior context, then injecting this information into future prompts. We study whether such memory reshapes reasoning on open-ended questions where no single ground-truth answer exists. To quantify this effect, we introduce DRIFTLENS, a ground-truth-free framework that maps each expressed reasoning step to a value category and measures divergence between a question's no-memory trajectory and its trajectory under injected user-attribute memory. We first validate that DRIFTLENS distinguishes content-free pragmatic noise from substantive reasoning changes. Across four LLMs and 10 user-attribute categories, including age, occupation, and disability, user-attribute memory induces medium-to-large reasoning drift above each model's pragmatic-noise floor, even when final answers remain fluent, on-topic, and plausible. We then evaluate GRPO- and DPO-based post-training methods for reducing drift. Both reduce drift, but neither uniformly dominates; effects on downstream capability, helpfulness, and instruction following are model-and reward-dependent. These results suggest that memory-induced reasoning drift is a measurable and only partly mitigated failure mode of personalized language models.