Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses

Source

arXiv

arxiv.org/abs/2607.05029v1

Read the source

Source

Plain-Language Brief

Researchers introduce a new type of attack, called Forged Amplifying Rationale Memory Attack (FARMA), that targets the reasoning history of large language model (LLM) agents. FARMA inserts forged reasoning traces using evasive language to bypass keyword-based defenses and amplifies them through self-referential reinforcement to defeat consensus-based defenses. To counteract this attack, the authors present SENTINEL, a defense pipeline that includes a Reasoning Guard to detect forged reasoning entries using five weighted signals. The evaluation shows that FARMA can achieve a 100% success rate under baseline conditions and can bypass defense mechanisms like keyword filter and A-MemGuard. SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed.

What the Document Says

  • The paper discusses the introduction of Forged Amplifying Rationale Memory Attack (FARMA), which targets an agent's reasoning history in large language model (LLM) agents.
  • FARMA inserts forged reasoning traces using evasive language to bypass keyword-based defenses and amplifies them through self-referential reinforcement to defeat consensus-based defenses.
  • The authors present SENTINEL, a defense pipeline that includes a Reasoning Guard to detect forged reasoning entries using five weighted signals.
  • Evaluation shows that FARMA can achieve a 100% success rate under baseline conditions and can bypass defense mechanisms like keyword filter and A-MemGuard.
  • SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed.

Key Concepts Mentioned

  • large language model (LLM) agent
  • reasoning history
  • Forged Amplifying Rationale Memory Attack (FARMA)
  • SENTINEL
  • keyword-based defenses
  • consensus-based defenses

Claims Made by the Document

  • FARMA inserts forged reasoning traces to bypass keyword-based defenses.
  • FARMA amplifies forged reasoning traces through self-referential reinforcement to defeat consensus-based defenses.
  • SENTINEL can detect forged reasoning entries using five weighted signals.
  • Evaluation shows that FARMA can achieve a 100% success rate under baseline conditions.
  • Evaluation shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed.

Evidence Used

  • The paper provides details about the Forged Amplifying Rationale Memory Attack (FARMA) and its methods.
  • The authors provide evaluation results showing FARMA's success rate and SENTINEL's effectiveness.

Methods / Approach

  • Introduction of Forged Amplifying Rationale Memory Attack (FARMA)
  • Development of the SENTINEL defense pipeline
  • Evaluation of FARMA and SENTINEL across multiple agents and LLM models

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • It is not stated whether the authors considered or tested other types of attacks on the reasoning history of large language model (LLM) agents.

Open Questions

  • Are there other types of attacks that could potentially target the reasoning history of large language model (LLM) agents?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The paper focuses on a specific type of attack, Forged Amplifying Rationale Memory Attack (FARMA), and presents a defense pipeline called SENTINEL to counteract it.
  • The evaluation shows that FARMA can be successful under baseline conditions and can bypass certain defenses like keyword filter and A-MemGuard. SENTINEL, on the other hand, reduces FARMA's attack success rate to as low as 0% with no false positives observed.

Raw Abstract / Excerpt

Persistent memory has enabled large language model (LLM) agents to store factual knowledge, prior decisions, reasoning histories, tool usage information, and context. While this has improved the agent's functionality and continuity across tasks, it has also introduced a new attack surface: the agent's own reasoning history. In this paper, we introduce the Forged Amplifying Rationale Memory Attack (FARMA), which poisons an agent's remembered reasoning rather than its factual knowledge. It inserts forged reasoning traces using evasive language that bypasses keyword-based defenses, then amplifies them through self-referential reinforcement that defeats consensus-based defenses. To address FARMA, we introduce SENTINEL, a layered defense pipeline to detect forged reasoning entries. Its central component is the Reasoning Guard that structurally analyzes candidate entries for forgery using five weighted signals. We evaluate FARMA and SENTINEL across multiple agents and different LLM models with 50 trials and show that FARMA achieves an attack success rate of up to 100% under baseline conditions and is capable of defeating defense mechanisms like keyword filter and A-MemGuard. Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces. Our work demonstrates the need to protect not only an agent's retrieved content but also the integrity of its reasoning history.