SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Jinwei Hu, Yi Dong, Youcheng Sun, Xiaowei Huang
• Venue: arXiv: cs.SE
• Published: 2026-07-02
• Collected: 2026-07-03
• URL: http://arxiv.org/abs/2607.02345v1
• PDF: https://arxiv.org/pdf/2607.02345v1
• DOI: UNKNOWN

Plain-Language Brief

The paper presents SkillFuzz, an execution-free testing approach that helps detect unintended objectives (implicit intents) in large language model (LLM)-based agents operating on open skill marketplaces. SkillFuzz identifies potential conflicts among skills by using contract-guided Monte Carlo Tree Search and discovers over 1,000 distinct implicit intents across various workloads, confirming more than 80% of high-risk compositions during execution-time validation.

What the Document Says

• SkillFuzz is an execution-free testing approach for detecting unintended objectives (implicit intents) in LLM-based agents using open skill marketplaces.
• The method uses structured skill contracts and contract-guided Monte Carlo Tree Search to prioritize potentially conflicting compositions.
• SkillFuzz discovers over 1,000 distinct implicit intents across representative skill-marketplace workloads.
• More than 80% of the highest-risk flagged compositions are confirmed during execution-time validation.
• SkillFuzz identifies more high-severity implicit intents than alternative search strategies while exploring only a fraction of the pairwise interaction space they require.

Key Concepts Mentioned

• Large Language Model (LLM)
• Open skill marketplaces
• Skill compositions
• Implicit intents
• Planning artifacts
• Deviations from a skill-free baseline
• Contract-guided Monte Carlo Tree Search

Claims Made by the Document

• SkillFuzz is an effective approach for detecting implicit intents in LLM-based agents on open skill marketplaces.
• SkillFuzz can confirm over 80% of high-risk flagged compositions during execution-time validation.

Evidence Used

• The paper presents SkillFuzz and its evaluation results across representative skill-marketplace workloads.

Methods / Approach

• SkillFuzz uses contract-guided Monte Carlo Tree Search to prioritize potentially conflicting compositions

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

• The paper does not specify the specific evaluation metrics used for measuring the effectiveness of SkillFuzz.
• It is unclear how generalizable SkillFuzz's performance is to other types of open skill marketplaces beyond those described in the study.

Open Questions

• What are the specific evaluation metrics used for measuring the effectiveness of SkillFuzz?
• How generalizable is SkillFuzz's performance to other types of open skill marketplaces?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The paper focuses on LLM-based agents operating in open skill marketplaces and the discovery of implicit intents through a fuzzing approach.

Raw Abstract / Excerpt

Large Language Model (LLM)-based agents increasingly automate software engineering tasks through reusable skills, natural-language instruction documents that guide planning and execution. Open skill marketplaces enable users to assemble agents by co-activating community-contributed skills, but marketplace operators typically audit skills in isolation. As a result, individually benign skills may interact to redirect an agent toward unintended objectives, which we term implicit intents. Detecting such intents is challenging because the effect emerges only through skill composition, execution environments are often unavailable at admission time, and the space of possible co-activations grows exponentially with marketplace size. In this paper, we formulate implicit-intent discovery as a fuzzing problem over skill compositions, where skill compositions are the unit under test, planning artifacts expose agent intent before execution, and deviations from a skill-free baseline serve as a differential oracle. Based on this formulation, we propose skillfuzz, the first execution-free testing approach that extracts structured skill contracts and uses contract-guided Monte Carlo Tree Search to prioritize potentially conflicting compositions. Across representative skill-marketplace workloads, skillfuzz discovers over 1,000 distinct implicit intents under a fixed query budget, confirms more than 80% of the highest-risk flagged compositions during execution-time validation, and identifies substantially more high-severity implicit intents than alternative search strategies while exploring only a fraction of the pairwise interaction space they require.