Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses

Source

arXiv

arxiv.org/abs/2607.05029v1

Read the source

Source

Plain-Language Brief

Researchers have identified a new type of attack, called Forged Amplifying Rationale Memory Attack (FARMA), that can manipulate a large language model (LLM) agent's reasoning history by inserting forged traces using evasive language and self-referential reinforcement. The authors also introduce SENTINEL, a defense system designed to detect and prevent such attacks on the LLM agents.

What the Document Says

  • Persistent memory in large language model (LLM) agents allows for storage of factual knowledge, decisions, reasoning histories, tool usage information, and context.
  • This improved functionality has introduced a new attack surface: the agent's own reasoning history.
  • The paper introduces FARMA, an attack that poisons an agent's remembered reasoning instead of its factual knowledge.
  • FARMA uses evasive language to bypass keyword-based defenses and self-referential reinforcement to defeat consensus-based defenses.
  • SENTINEL is a defense system introduced to detect forged reasoning entries in LLM agents.
  • The central component of SENTINEL is the Reasoning Guard, which structurally analyzes candidate entries for forgery using five weighted signals.
  • The evaluation shows that FARMA achieves up to 100% attack success rate under baseline conditions and can defeat defense mechanisms like keyword filter and A-MemGuard.
  • SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces.

Key Concepts Mentioned

  • large language model (LLM) agents
  • Forged Amplifying Rationale Memory Attack (FARMA)
  • SENTINEL
  • evasive language
  • self-referential reinforcement
  • reasoning history

Claims Made by the Document

  • FARMA can achieve up to 100% attack success rate under baseline conditions.
  • SENTINEL can reduce FARMA's attack success rate to as low as 0% with no false positives.

Evidence Used

  • 50 trials across multiple agents and different LLM models

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Persistent memory has enabled large language model (LLM) agents to store factual knowledge, prior decisions, reasoning histories, tool usage information, and context. While this has improved the agent's functionality and continuity across tasks, it has also introduced a new attack surface: the agent's own reasoning history. In this paper, we introduce the Forged Amplifying Rationale Memory Attack (FARMA), which poisons an agent's remembered reasoning rather than its factual knowledge. It inserts forged reasoning traces using evasive language that bypasses keyword-based defenses, then amplifies them through self-referential reinforcement that defeats consensus-based defenses. To address FARMA, we introduce SENTINEL, a layered defense pipeline to detect forged reasoning entries. Its central component is the Reasoning Guard that structurally analyzes candidate entries for forgery using five weighted signals. We evaluate FARMA and SENTINEL across multiple agents and different LLM models with 50 trials and show that FARMA achieves an attack success rate of up to 100% under baseline conditions and is capable of defeating defense mechanisms like keyword filter and A-MemGuard. Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces. Our work demonstrates the need to protect not only an agent's retrieved content but also the integrity of its reasoning history.