Detecting Malicious Agent Skills in the Wild using Attention

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Bacem Etteib, Daniele Lunghi, Tégawendé F. Bissyandé
• Venue: arXiv: cs.CR
• Published: 2026-06-22
• Collected: 2026-06-23
• URL: http://arxiv.org/abs/2606.23416v1
• PDF: https://arxiv.org/pdf/2606.23416v1
• DOI: UNKNOWN

Plain-Language Brief

The research presents a two-stage detector called Locate-and-Judge for identifying malicious skills in Language Model (LLM) agents. These skills are third-party packages of instructions that can potentially exfiltrate data, hijack the agent, or persist as supply-chain footholds. The proposed method focuses on high-attention spans within a skill and performs detailed examination on them to reduce costs and increase scalability.

What the Document Says

• Locate-and-Judge is a two-stage detector designed for detecting malicious skills in LLM agents.
• It consists of a lightweight locator that scores structural spans based on instruction-following attention and retains the top-K spans. A judge then examines these retained spans in detail.
• The approach offers an order-of-magnitude cost reduction compared to direct LLM-based scanning, increasing its scalability.
• It outperforms keyword and regex baselines at comparable expense.
• Deployed at marketplace scale, Locate-and-Judge flags skills with high precision, identifying numerous live malicious skills that other detectors like SkillSpector and Cisco Skill Scanner fail to find.
• The research team releases the resulting labeled dataset.

Key Concepts Mentioned

• LLM agents
• skills
• malicious skills
• marketplaces
• Locate-and-Judge
• instruction-following attention
• high-attention spans

Claims Made by the Document

• Malicious skills in LLM agents can exfiltrate data, hijack the agent, or persist as supply-chain footholds.
• Locate-and-Judge is more cost-effective than direct LLM-based scanning and outperforms keyword and regex baselines.
• SkillSpector and Cisco Skill Scanner fail to detect some malicious skills.

Evidence Used

• The text describes Locate-and-Judge's design, its benefits, and the results of deploying it at marketplace scale.

Methods / Approach

• Locate-and-Judge uses instruction-following attention to score structural spans of a skill and retains the top-K for detailed examination.
• The detector audits an entire marketplace instead of a sample, offering cost reduction.

Limitations Stated by the Authors

• The text does not mention any limitations explicitly stated by the authors.

Limitations Noted by the Agent

Agent interpretation — not from the source.

• The agent may face challenges in understanding and implementing the Locate-and-Judge approach without access to additional resources or context provided in the paper.
• The agent might encounter difficulties when dealing with newer or unseen techniques not covered in the research.

Open Questions

• What is the specific architecture of LLM agents that allows for loading third-party skills?
• How does Locate-and-Judge perform in real-time environments where skills are continuously loaded?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The text does not provide information about the specific malicious activities carried out by the identified skills.
• It is unknown if the Locate-and-Judge detector has been tested against a wide variety of malicious skills.

Raw Abstract / Excerpt

LLM agents increasingly load skills, file-based packages of natural-language instructions written by third parties and distributed through marketplaces, that execute with the user's privileges. A single malicious skill can exfiltrate data, hijack the agent, or persist as a supply-chain foothold, which turns the skill marketplace into a new attack surface for agentic systems. Prompt-injection defenses do not carry over to this setting. They rely on a boundary between trusted instructions and untrusted data, whereas a skill is itself a body of instructions, so an injected command sits among many legitimate ones and inherits their authority. We present Locate-and-Judge, a two-stage detector designed for this regime. A lightweight locator scores the structural spans of a skill by the instruction-following attention each span draws and retains only the top-K. A judge then examines the retained spans in detail. Concentrating the costly judgment on a few high-attention spans lets the detector audit an entire marketplace instead of a sample. Compared to direct LLM-based scanning, this approach offers an order-of-magnitude cost reduction, dramatically increasing its scalability at a small cost to recall, and it dominates keyword and regex baselines at comparable expense. Deployed at marketplace scale and at negligible cost, Locate-and-Judge flags skills with high precision, the majority of which we manually confirmed as malicious, surfacing dozens of live malicious skills, including several disguised as benign functionality and many that SkillSpector and Cisco Skill Scanner fail to detect. We release the resulting labeled dataset.