Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems

Source

arXiv

arxiv.org/abs/2606.31639v1

Read the source

Source

Plain-Language Brief

This research paper discusses the increased role of large language models in various systems beyond just text generation, and how this shift changes the security landscape. The authors systematize the existing literature on vulnerabilities in these systems by examining eight stages in the lifecycle and application stack. They analyze attacker capabilities, affected security objectives, representative attacks, practical risks, evaluation practices, and defenses for each stage. The paper also identifies specific LLM-related vulnerabilities related to various security objectives like confidentiality, integrity, availability, etc., and proposes a research agenda for secure LLM systems.

What the Document Says

  • Large language models are used in various applications beyond text generation.
  • The paper systematizes literature on vulnerabilities in large language model systems through a lifecycle and application-stack lens.
  • There are eight stages of attack identified: data collection, pretraining, post-training alignment, model packaging and supply chain, retrieval and memory, prompting and inference, tool/agent execution, and deployment/maintenance.
  • Attacker capabilities, affected security objectives, representative attacks, practical risks, evaluation practices, and defenses are analyzed for each stage.
  • The paper identifies specific LLM-related vulnerabilities related to various security objectives like confidentiality, integrity, availability, safety, privacy, fairness, accountability, and agency-control.

Key Concepts Mentioned

  • Large language models
  • Security vulnerabilities
  • Lifecycle
  • Application stack
  • Attacks
  • Risks
  • Defenses
  • Confidentiality
  • Integrity
  • Availability
  • Safety
  • Privacy
  • Fairness
  • Accountability
  • Agency-control

Claims Made by the Document

  • Large language models pose new security risks due to their expanded role.
  • The proposed systematization emphasizes where trust boundaries fail in LLM systems.

Evidence Used

  • The paper discusses the increasing use of large language models in various applications beyond text generation.
  • The authors organize attacks across eight stages and analyze various aspects for each stage.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Large language models are no longer only text generators. They are increasingly embedded in retrieval pipelines, enterprise assistants, coding environments, robotic systems, security-operation workflows, and autonomous agents that can read private data, call tools, write files, execute code, and act across organizational boundaries. This shift changes the security problem: risks do not arise from the model weights alone, but from the full lifecycle and application stack through which data, prompts, model outputs, tools, memories, and user authority interact. This paper systematizes the literature on vulnerabilities in large language model systems through a lifecycle and application-stack lens. We organize attacks across eight stages: data collection, pretraining, post-training alignment, model packaging and supply chain, retrieval and memory, prompting and inference, tool/agent execution, and deployment/maintenance. For each stage, we analyze attacker capabilities, affected security objectives, representative attacks, practical risks, evaluation practices, and defenses. We further map LLM-specific vulnerabilities to confidentiality, integrity, availability, safety, privacy, fairness, accountability, and agency-control objectives. Unlike taxonomies that list isolated attack names, the proposed systematization emphasizes where trust boundaries fail, how untrusted data becomes executable instruction, how delegated authority amplifies model errors, and why point defenses rarely compose. We close with a research agenda for secure LLM systems, including compositional security, provenance-aware retrieval, tool-call containment, long-horizon agent evaluation, privacy-preserving adaptation, realistic red teaming, and deployment-grade incident response.