When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Jiawei Liu, Jiacheng Guo, Tian Zhang, Yiwei Xu, Juan Wang, Jinlin Fan, Bowen Xiao, Chi Guo, Keyan Guo, Hongxin Hu
• Venue: arXiv: cs.RO
• Published: 2026-08-17
• Collected: 2026-08-18
• URL: http://arxiv.org/abs/2608.16806v1
• PDF: https://arxiv.org/pdf/2608.16806v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

The text discusses the evolution of Large Language Models (LLMs) from text generation models to core components of agents that can perceive environments, invoke tools, and execute tasks. These LLM-driven embodied agents are further integrated with robotic systems, allowing them to perform functions like task understanding, high-level planning, and behavioral decision-making. The text mentions several examples such as SayCan, Code as Policies, ProgPrompt, VoxPoser, PaLM-E, RT-2, and GR00T N1.

What the Document Says

• LLMs have capabilities in in-context learning, task decomposition, step-by-step reasoning, and code generation.
• Traditional LLM Agents obtain information through webpages, documents, databases, or external tools and generate corresponding invocation sequences according to user goals.
• Integrating LLMs with robotic systems allows them to perform functions like task understanding, high-level planning, and behavioral decision-making.
• SayCan combines the task reasoning capability of language models with the affordances of robotic skills.
• Code as Policies and ProgPrompt generate robot task plans through policy code and programmatic prompting respectively.
• VoxPoser uses language models and vision-language models to construct three-dimensional value maps to guide robotic manipulation.
• Vision-language-action models such as PaLM-E, RT-2, and GR00T N1 further strengthen the connection among language, visual perception, and robotic actions.

Key Concepts Mentioned

• Large Language Models (LLMs)
• Embodied Agents
• Robotic Systems
• Task Understanding
• High-Level Planning
• Behavioral Decision-Making
• SayCan
• Code as Policies
• ProgPrompt
• VoxPoser
• PaLM-E
• RT-2
• GR00T N1

Claims Made by the Document

• LLMs have capabilities in in-context learning, task decomposition, step-by-step reasoning, and code generation.

Evidence Used

• Not stated in source

Methods / Approach

• Not stated in source

Limitations Stated by the Authors

The text does not mention any limitations explicitly attributed to the authors.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The potential risks or limitations of integrating LLMs with robotic systems for autonomous decision-making and manipulation are not discussed.

Open Questions

• What are the potential risks or limitations of integrating LLMs with robotic systems?
• How does the model understand user instructions in the context of LLM-driven embodied agents?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The text focuses on the capabilities and examples of LLM-driven embodied agents, but it does not discuss the potential risks or limitations associated with this technology.

Raw Abstract / Excerpt

Large Language Models (LLMs) have demonstrated capabilities in in-context learning, task decomposition, step-by-step reasoning, and code generation, driving their gradual evolution from text generation models into the core of agents capable of perceiving environments, invoking tools, and executing tasks. Traditional LLM Agents typically obtain information through webpages, documents, databases, or external tools and generate corresponding invocation sequences according to user goals; when this technology is further integrated with robotic systems, large language models begin to undertake functions such as task understanding, high-level planning, and behavioral decision-making. SayCan combines the task reasoning capability of language models with the affordances of robotic skills, while Code as Policies and ProgPrompt generate robot task plans through policy code and programmatic prompting, respectively, and VoxPoser uses language models and vision-language models to construct three-dimensional value maps to guide robotic manipulation \cite{6,7,8,9}. Vision-language-action models such as PaLM-E, RT-2, and GR00T N1 further strengthen the connection among language, visual perception, and robotic actions \cite{10,11,12}. In such LLM-driven embodied agents, the model not only needs to understand user instructions, but also needs to combine scene states, object attributes, spatial relations, and execution feedback to complete task grounding, and then hand the generated action plan to skill libraries, motion planners, or controllers for execution.