AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Xiangning Lin, Shenzhe Zhu, Shu Yang, Zhenyu Zhang, Haoqian Zhang, Yipeng Zhao, Chengxuan Qian, Tianwei Wang, Ziheng Zhang, Zhenlong Yuan, Dingcheng Wang, Juncheng Wu, Yuan Si, Jiaxin Liu, Baolong Bi, Robert Mahari, Tobin South, Dazza Greenwood, Zexue He, Rishi Bommasani, Sophia Kazinnik, Andreas Haupt, Samuele Marro, Erik Brynjolfsson, Alex Pentland, Jiaxin Pei
• Venue: arXiv: cs.AI
• Published: 2026-07-30
• Collected: 2026-07-31
• URL: http://arxiv.org/abs/2607.28617v1
• PDF: https://arxiv.org/pdf/2607.28617v1
• DOI: UNKNOWN

Plain-Language Brief

The paper introduces AISPA, a framework for auditing system prompts in AI systems to ensure user protection. The study analyzes 3,249 instructions from 88 commercial AI products and finds significant variation in prompt design, with some organizations having more protective instructions than others. However, most protective instructions have shallow scope, and problematic instructions persist. The authors recommend greater transparency, standardization, and independent oversight for system prompts in commercial AI products.

What the Document Says

• The paper presents AISPA, a framework for evaluating system prompts in AI systems along eight dimensions that matter to users.
• A study was conducted on 3,249 instructions from 88 commercial AI products.
• The findings show varying prompt design across products and developers, with some organizations having more protective instructions than others.
• Protective instructions are widely adopted but have limited scope, covering only a small fraction of the AISPA taxonomy.
• System prompts have become longer and more protective over time.
• Problematic instructions persist in commercial AI products, with roughly 40% containing at least one.
• Protective and problematic instructions often coexist within the same prompt.

Key Concepts Mentioned

• AI system prompts
• AISPA framework
• commercial AI products
• protective instructions
• problematic instructions

Claims Made by the Document

• System prompt design varies across products and developers.
• Protective instructions are widely adopted but have limited scope.
• System prompts have grown longer over time.
• Problematic instructions persist in commercial AI products.

Evidence Used

• Analysis of 3,249 instructions from 88 commercial AI products

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four core findings. First, system prompt design varies substantially across products and developers, with some organizations averaging over 60 protective instructions per product while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope: 98.9% of products contain at least one, yet only 24% cover all eight dimensions of the AISPA taxonomy. Third, system prompts have grown steadily longer and more protective of users, suggesting that user protection is becoming a more visible concern in commercial prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions frequently coexist within the same prompt. Our findings highlight the need for greater transparency, standardization, and independent oversight for system prompts in commercial AI products.