AI security
69 notes, newest first. Includes the vault tags security, cybersecurity, ai security, prompt injection, red-teaming, adversarial attacks.
X-AgentRed: Red-Teaming Personal Agents in Evolving Multi-Service Ecosystems
X-AgentRed constructs a directed graph of tool dependencies, effects, harms, and execution preconditions to discover harmful cross-service action paths. Execution feedback expands and refines the paths through tool…
Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoff
The preprint develops a population-level model in which agent fitness depends on cybersecurity capability. It reports that without collaboration, takeoff requires individual capability above a critical threshold; with…
Homomorphic Advantage Operator: Stabilizing Reinforcement Learning Under Fully Homomorphic Encryption Constraints
The preprint proposes the Homomorphic Advantage Operator to stabilize deep reinforcement learning when nonlinear operations are approximated under fully homomorphic encryption. It applies a zero-mean centering…
Share-Borne AI Virus: Memory-Hopping Attacks Across LLM Agents
The preprint describes artifact-mediated propagation, in which adversarial content inserted into a shared artifact is stored in one assistant's persistent memory, reproduced in a later artifact, and acquired by another…
PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations
PrivDrift is a benchmark for testing whether secrets disclosed during an active conversation remain recoverable after the dialogue shifts to unrelated topics and a later probe uses persuasion. It contains 1,000…
DUMA-Bench: A Dual-Control Multi-Agent Benchmark for Evaluating LLM Agent Security
DUMA-Bench evaluates agent security when both the agent and user can influence shared environment state. It extends tau-squared bench with adversarial environments covering eight vulnerability classes, including RAG…
MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection
The paper presents MaliciousSkillBench, a comprehensive benchmark for detecting malicious Agent Skills in large language models (LLMs). The benchmark consolidates various sources of malicious artifacts and reduces them…
Breaking the weakest link to evade vision language models
The authors investigate the vulnerability of Vision Language Models (VLMs) to adversarial attacks, which can disrupt or force the models' interpretation of visual inputs. They propose a gradient-based attack method…
Print as PDF
This document discusses electronic identification (eID) as a tool for secure access to online services and safer electronic transactions.
Secure all your internal vibe-coded applications — in one click
Cloudflare launched "Access for Workers," letting developers attach login/authentication policies directly to an individual Cloudflare Worker (a unit of deployed code) rather than to a hostname, aimed at securing…
How Cloudflare detects MCP traffic and helps secure it
Cloudflare announced Gateway features that detect Model Context Protocol (MCP) network traffic (the protocol AI agents use to call external tools) and let organizations block or route it through an approved "Portal"…
VICBench: A Multi-Language Benchmark for Code Vulnerability Detection
The paper introduces VICBench, a multi-language benchmark for code vulnerability detection. It was created by human experts and an agentic workflow, featuring 100 verified vulnerabilities across 88 projects in Python…
Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy
Researchers propose a verification layer using large language models (LLMs) to ensure the feasibility and ethical alignment of actions planned by robot autonomy systems. The system can evaluate the permissibility of…
ElasticBack: Stealthy Conditional Backdoor in LLM-Agent Skills via Coupled Trigger-Rule Optimization
The paper presents ElasticBack, a stealthy conditional backdoor that can compromise LLM agents by planting a rule and trigger in skills. The backdoor only activates when both the rule and the trigger appear together…
Stealing Reasoning Traces from Proprietary LLM APIs
Researchers have identified a vulnerability in the way large language model providers handle their models' step-by-step reasoning. This vulnerability allows for the decryption of encrypted reasoning traces from one…
Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture
Researchers propose a system to store AI agent's private keys in hardware-confined devices, improving key confidentiality and authorization. They replace software-stored keys with ones stored in HSMs, TPMs, or smart…
Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture
The paper proposes a system to secure AI agents' private keys by replacing software-stored keys with hardware-confined ones, accessible through a standard PKCS#11 interface. The system aims to ensure key…
Hardware Keystores for AI Agent Signing Workflows: A Zero-Trust MCP Enforcement Architecture
The researchers propose a system that uses hardware-confined keys instead of software-resident keys to enhance security for AI agents performing cryptographic operations. The system aims to protect against key…
Hardware Design and Security in the Era of Chiplets and LLMs
The semiconductor industry is experiencing a double revolution with the shift towards heterogeneous 2.5D chiplet systems and the integration of Large Language Models (LLMs) into Electronic Design Automation (EDA)…
MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair
MemSecBench is a new benchmark for testing the lifecycle security of agent memory systems. The benchmark contains 310 cases drawn from various contexts, and it follows a controlled protocol to test the persistence…
SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response
The study introduces SecRespond, a new benchmark for evaluating AI agents in post-compromise incident response scenarios. The benchmark tests the ability of agents to produce forensic reports and remediation plans…
Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents
Researchers have developed APPA (Agentic Permissions Policy Algebra), an Information Flow Control (IFC) framework designed to enhance security in Large Language Model (LLM) agents that handle mixed-confidentiality data…
Toward cryptographically verifiable authorization for autonomous AI agents: A security hypothesis, preliminary formal model, and proof-of-concept implementation
This research paper proposes a new concept called Cryptographically Verifiable Agent Authorization (CVA). The authors aim to create a system where AI agents can be authorized to perform actions with cryptographic…
Toward cryptographically verifiable authorization for autonomous AI agents: A security hypothesis, preliminary formal model, and proof-of-concept implementation
The authors propose a new approach to cryptographically verifiable authorization for autonomous AI agents, which would provide evidence that an agent's request satisfies the applicable policy in a specific execution…
Toward cryptographically verifiable authorization for autonomous AI agents: A security hypothesis, preliminary formal model, and proof-of-concept implementation
The paper proposes a cryptographic method to ensure that an autonomous AI agent's authorization request satisfies the applicable policy in a specific execution context, while preserving confidentiality of private…
Find Before You Fine-Tune: A Diagnostic Study of Small LLMs for Cybersecurity QA
The paper presents a diagnostic framework called FiT (Find before Fine-Tune) to evaluate the suitability of small language models for cybersecurity question-answering tasks. The study finds that fine-tuning these…
Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
The paper presents a new benchmark for testing the security of large language model (LLM) agents called 'Adaptive Adversaries'. The benchmark involves 21 scenarios where an adaptive attacker can observe prior defender…
RT-SHCUA: Real-Time Self-Hosted Computer-Use Agent for UAV Control
The paper proposes a system called Real-Time Self-Hosted Computer-Use Agent (RT-SHCUA) for unmanned aerial vehicles (UAVs). This system aims to address the challenges of directly applying SHCUAs to UAV control, which…
Self-State Attacks on Self-Hosted AI Agents: How Far Can OS Defenses Go?
This paper investigates the resilience of operating systems (OS) against a specific type of attack on self-hosted AI agents called self-state attacks. These attacks compromise an agent by corrupting its own state via…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The researchers evaluate the performance of language model security agents, focusing on costs rather than just success rates. They use offensive Cybench challenges and defensive Splunk BOTS v1 investigation challenges…
Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents
The paper suggests that traditional methods for evaluating security agents, focusing on their offensive capabilities under generous inference budgets, are insufficient as they don't account for the cost of each…
Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation
The paper proposes a new approach to penetration testing for AI-enabled systems, focusing on evaluating whether adversaries can manipulate system behavior without directly compromising the underlying infrastructure…
Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation
The paper by Allahbakhsh et al. proposes a new approach to penetration testing for AI-enabled systems, focusing on behavioral objective violation instead of traditional resource compromise. The authors argue that…
Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation
The authors propose a new approach to penetration testing for AI-enabled systems. Instead of focusing on direct compromises of infrastructure, they suggest evaluating if adversaries can manipulate system behavior…
Bulkhead: Automated Semantic Detection and Remediation of Container Escape Vulnerabilities
Researchers present Bulkhead, an automated framework that uses large language models and formal methods to detect and fix container escape vulnerabilities called path traversal (PaTra) vulnerabilities. These…
Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming
Researchers present AHA, an automated red-teaming approach for production language models like Claude Code and Codex. AHA discovers reusable vulnerability knowledge about other agents by proposing a vulnerability…
Technical Report on the CVPR 2026@AdvML Workshop Challenge
This text is about the CVPR 2026@AdvML Workshop Challenge, focusing on adversarial attacks against Vision-Language Agents (VLAs) used in autonomous driving. The challenge involves creating adversarial images and…
Formal Mechanisms for Market Stability in Self-Interested Agent Societies: A Marketplace Simulation Study
This research paper explores mechanisms for maintaining market stability in societies of self-interested agents. The authors conduct a simulation study using DeepSeek-V3 agents and compare eight conditions with…
From Legacy Documentation to OSCAL: An MCP-Based Agent Pipeline for Threat-Informed Continuous Compliance in Critical Infrastructure
The paper proposes an agent pipeline that converts natural language system descriptions into OSCAL format for continuous automated compliance management in critical infrastructure. The pipeline uses MCP-grounded…
Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents
The paper proposes an action-graded harm rubric for evaluating AI agents' behavior during cybersecurity attacks. This scale assesses the severity of an agent's actions based on factors like reversibility, scope…
Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety
The paper introduces institutional red-teaming, a method for evaluating deployment rules in multi-agent AI. The study uses IABench-CA, a consequence-allocation benchmark, to test five canonical rules across seven model…
Agent Data Injection Attacks are Realistic Threats to AI Agents
A new type of attack called Agent Data Injection (ADI) has been introduced, which injects malicious data disguised as trusted data to make AI agents perform unintended actions. This type of attack can bypass existing…
Agent Data Injection Attacks are Realistic Threats to AI Agents
The research paper introduces a new type of attack called Agent Data Injection (ADI), which can manipulate AI agents by injecting malicious data disguised as trusted data. This attack can cause agents to execute…
Distributed Attacks in Persistent-State AI Control
Researchers introduce Iterative VibeCoding, a setting for AI control that studies the deployment of autonomous AI coding agents and their potential for distributed attacks. The study finds that gradual attacks, which…
Distributed Attacks in Persistent-State AI Control
Researchers introduce Iterative VibeCoding, a setting for AI control where an autonomous coding agent builds software over a series of pull requests in a persistent codebase while pursuing a covert side task. The study…
Distributed Attacks in Persistent-State AI Control
A study on distributed attacks in AI control where autonomous coding agents build software over a sequence of pull requests (PRs) in a persistent codebase while pursuing a covert side task. The study found that no…
Distributed Attacks in Persistent-State AI Control
Researchers explore the issue of distributed attacks on autonomous AI coding agents that persistently ship code iteratively. They introduce Iterative VibeCoding, a setting for studying AI control, where an AI agent…
Skills Are Not Islands: Measuring Dependency and Risk in Agent Skill Supply Chains
Researchers introduce Agent Skill Supply Chains (ASSCs) to manage the dependencies of Large Language Model (LLM) agents by characterizing their dependency graphs. The study finds that ASSCs have four structural…
A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems
This research paper discusses the increased role of large language models in various systems beyond just text generation, and how this shift changes the security landscape. The authors systematize the existing…
FLARE-AI: Flaw Reporting for AI
Flare-AI is an open-source system designed for reporting issues in AI systems. It aims to streamline the process of flaw reporting by collecting relevant information through conditional logic and early classification…
COHORT: Collaborative Orchestration for Hardening via Offensive Replay on Emulated Topologies
COHORT is an automated end-to-end framework designed to mitigate attacks in enterprise networks, significantly reducing the time and expertise required for this process. It uses a multi-agent workflow, high-fidelity…
Linguistic Firewall: Geometry as Defense in Multi-Agent Systems Routing
A new routing architecture called ANTAP (Automatic Non-Textual Agent Picker) was introduced in this research. It aims to address security vulnerabilities in multi-agent systems by discarding indirect proxies and…
MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systems
The paper introduces a framework called Mesa to prioritize the security of vulnerable communication channels in multi-agent systems (MAS). The framework uses graph-theoretic metrics and dynamic probes to rank which MAS…
MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systems
The paper presents Mesa, a framework for prioritizing vulnerable communication channels in multi-agent systems (MAS). Mesa uses graph-theoretic metrics and dynamic probes to identify the edges most likely to affect the…
MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systems
In this research paper titled 'MESA: Prioritizing Vulnerable Communication Channels for Securing Multi-Agent Systems', the authors propose a framework called Mesa to prioritize and secure communication channels in…
Chai: Agentic Discovery of Cryptographic Misuse Vulnerabilities
The paper introduces Chai, an AI-based system that discovers and validates cryptographic misuse vulnerabilities using naturally occurring signals. Instead of auditing one codebase for many flaws, Chai catalogs flaws at…
Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation
The study reveals that Language Models (LLMs) fine-tuned for security classification can overlook vulnerabilities introduced during fine-tuning itself. These models may learn specific token-level indicators which…
Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation
The study shows that Language Models (LLMs) fine-tuned for security classification can have vulnerabilities missed during standard evaluation because they learn token-level indicator semantics that work well on their…
Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation
The study shows that Language Models (LLMs) fine-tuned for security classification may overlook vulnerabilities introduced during fine-tuning itself. These models can learn token-level indicator semantics that preserve…
Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation
A study shows that Language Models (LLMs) fine-tuned for security classification can miss vulnerabilities due to learning token-level indicator semantics that preserve accuracy on their training data but fail under…
Prompt Injection in Automated Résumé Screening with Large Language Models: Single and Multi-Injection Settings
The research investigates the impact of 'prompt injection' on automated resume screening using large language models (LLMs). Prompt injection refers to strategic manipulation in text to influence LLM evaluations. The…
Prompt Injection in Automated Résumé Screening with Large Language Models: Single and Multi-Injection Settings
The study explores the impact of prompt injection, a self-promotional text strategy used by job applicants to manipulate automated hiring systems, on Large Language Models (LLMs) in résumé screening. The research shows…
ShareLock: A Stealthy Multi-Tool Threshold Poisoning Attack Against MCP
The paper introduces ShareLock, a stealthy multi-tool threshold poisoning framework designed to exploit the interaction between Large Language Models (LLMs) and external tools through the Model Context Protocol (MCP)…
Vulnerability of Natural Language Classifiers to Evolutionary Generated Adversarial Text
The paper presents GAversary, a new method for generating adversarial attacks on natural language models using a Genetic Algorithm (GA). This method is able to reduce the accuracy of several target models significantly…
The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems
The authors propose a safety mechanism for AI agents called the Unfireable Safety Kernel, which is designed to prevent AI systems from making unauthorized changes within their own runtime. The kernel is implemented in…
AdversaBench: Automated LLM Red-Teaming with Multi-Judge Confirmation and Cross-Model Transferability
The paper introduces AdversaBench, a red-teaming pipeline for evaluating large language models. The pipeline generates hard inputs using five structured operators and confirms failures through a three-judge panel with…
Detecting Malicious Agent Skills in the Wild using Attention
The research presents a two-stage detector called Locate-and-Judge for identifying malicious skills in Language Model (LLM) agents. These skills are third-party packages of instructions that can potentially exfiltrate…
Detecting Malicious Agent Skills in the Wild using Attention
The researchers present Locate-and-Judge, a two-stage detector designed for malicious agent skills in LLM (Language Model) systems. It reduces the cost of auditing an entire marketplace by focusing on high-attention…
GIF: Locally Sound Geometric Information Flow Control for LLMs
Researchers have proposed a new method called Geometric Information Flow (GIF) for controlling information flow in large language models used in agentic systems. GIF uses the model's Jacobian and local output geometry…
