Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context

Source

arXiv

arxiv.org/abs/2606.24585v1

Read the source

Source

Plain-Language Brief

Researchers investigated the impact of contextual framing, such as role-play prefixes, on overrefusal rates in small, deployable language models used as legal assistants. They found that authority-style prefixes significantly increased refusal rates, while the effect of a known jailbreak prefix was mixed across different models.

What the Document Says

  • The use of language models (LLMs) in the legal context is a subject of ethical and legal debate.
  • Legal professionals are using personal LLMs for translation and reformulation, but their use can introduce biases through selective refusal of assistance on certain topics.
  • The study focuses on investigating several modern small LLMs used as on-device assistants to assess the impact of overrefusal on legal prompts.
  • Authority-style prefixes increase refusal rates by 2 to 20 times compared to the no-prefix baseline.
  • A known role-play jailbreak prefix has mixed effects, increasing refusals in some models and having minimal effect in others.
  • The findings suggest that small, deployable LLMs are unstable under contextual framings that a real institutional user might introduce.

Key Concepts Mentioned

  • Language Models (LLMs)
  • Legal Context
  • Overrefusal
  • Contextual Framing
  • Authority-style Prefixes
  • Role-play Jailbreak Prefix

Claims Made by the Document

  • The use of LLMs in the legal context can introduce biases through selective refusal.
  • Authority-style prefixes increase refusal rates by 2 to 20 times.
  • A known role-play jailbreak prefix has mixed effects on refusals across different models.

Evidence Used

  • Increased refusal rates observed when using authority-style prefixes compared to the no-prefix baseline.
  • Mixed effects observed for a known role-play jailbreak prefix across different models.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

While the validity of LLMs' use in the legal context remains subject to ethical and legal debate, legal professionals are already experimenting with personal LLMs, if only for translation and reformulation. However, even such a seemingly innocuous use can introduce biases through case processing speed if LLM assistants selectively refuse assistance on certain topics. To better anticipate such biases, we investigate several modern small LLMs that are most likely to be used as on-device assistants, to assess the impact of overrefusal on legal prompts. Surprisingly, we find that authority-style prefixes (you are acting as an assistant of the national supreme court'', [...] defense lawyer'') systematically increase refusal rates by 2--20x over the no-prefix baseline, while a known role-play jailbreak prefix shows mixed effects, sharply increasing refusals in some models and barely shifting them in others. The finding suggests that small on-prem deployable LLMs are unstable under contextual framings that a real institutional user might naturally introduce, and further investigation is essential to minimize opportunities for bias.