Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces

Source

arXiv

arxiv.org/abs/2607.02345v1

Read the source

Source

Plain-Language Brief

The paper proposes SkillFuzz, a method to detect potential unintended objectives (implicit intents) in Large Language Model (LLM)-based agents that use reusable skills from open skill marketplaces. SkillFuzz identifies these implicit intents by treating the discovery of implicit intents as a fuzzing problem over skill compositions, and uses contract-guided Monte Carlo Tree Search to prioritize potentially conflicting compositions.

What the Document Says

  • The paper discusses Large Language Model (LLM)-based agents that use reusable skills from open skill marketplaces.
  • It mentions a challenge where individual benign skills may interact to redirect an agent towards unintended objectives, which are referred to as implicit intents.
  • SkillFuzz is proposed as a solution for detecting these implicit intents, treating the problem as a fuzzing problem over skill compositions.
  • The method uses structured skill contracts and contract-guided Monte Carlo Tree Search to prioritize potentially conflicting compositions.
  • According to the paper, SkillFuzz can discover over 1000 distinct implicit intents under a fixed query budget.
  • It also confirms more than 80% of the highest-risk flagged compositions during execution-time validation.

Key Concepts Mentioned

  • Large Language Model (LLM)
  • Open skill marketplaces
  • Skill composition
  • Implicit intents
  • Fuzzing
  • Structured skill contracts
  • Contract-guided Monte Carlo Tree Search

Claims Made by the Document

  • Individual benign skills may interact to redirect an agent towards unintended objectives.
  • SkillFuzz can discover over 1000 distinct implicit intents under a fixed query budget.
  • SkillFuzz confirms more than 80% of the highest-risk flagged compositions during execution-time validation.

Evidence Used

  • The paper presents SkillFuzz as a solution for detecting implicit intents in LLM-based agents.
  • Results from representative skill-marketplace workloads are provided to support the claims about SkillFuzz's performance.

Methods / Approach

  • SkillFuzz treats the discovery of implicit intents as a fuzzing problem over skill compositions.
  • It uses structured skill contracts and contract-guided Monte Carlo Tree Search for prioritization.

Limitations Stated by the Authors

  • The effect of implicit intents emerges only through skill composition.
  • Execution environments are often unavailable at admission time.
  • The space of possible co-activations grows exponentially with marketplace size.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The paper does not provide real-world implementation details or evaluation in a dynamic environment.

Open Questions

  • How effective is SkillFuzz in detecting implicit intents in real-world scenarios with a dynamic environment?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The paper focuses on the problem of implicit intents in open skill marketplaces for LLM-based agents.
  • SkillFuzz is proposed as a solution to detect these implicit intents by treating the problem as a fuzzing problem over skill compositions.

Raw Abstract / Excerpt

Large Language Model (LLM)-based agents increasingly automate software engineering tasks through reusable skills, natural-language instruction documents that guide planning and execution. Open skill marketplaces enable users to assemble agents by co-activating community-contributed skills, but marketplace operators typically audit skills in isolation. As a result, individually benign skills may interact to redirect an agent toward unintended objectives, which we term implicit intents. Detecting such intents is challenging because the effect emerges only through skill composition, execution environments are often unavailable at admission time, and the space of possible co-activations grows exponentially with marketplace size. In this paper, we formulate implicit-intent discovery as a fuzzing problem over skill compositions, where skill compositions are the unit under test, planning artifacts expose agent intent before execution, and deviations from a skill-free baseline serve as a differential oracle. Based on this formulation, we propose skillfuzz, the first execution-free testing approach that extracts structured skill contracts and uses contract-guided Monte Carlo Tree Search to prioritize potentially conflicting compositions. Across representative skill-marketplace workloads, skillfuzz discovers over 1,000 distinct implicit intents under a fixed query budget, confirms more than 80% of the highest-risk flagged compositions during execution-time validation, and identifies substantially more high-severity implicit intents than alternative search strategies while exploring only a fraction of the pairwise interaction space they require.