legal ai
4 notes, newest first.
ContractScrub: A benchmark for final review of legal contracts
The paper introduces ContractScrub, a new benchmark for evaluating the performance of Language Models (LLMs) in reviewing legal contracts for errors. The study finds that while LLMs perform well on general benchmarks…
InsufficiencyBench: Evaluating LLM legal advice on underspecified user queries
Researchers introduce InsufficiencyBench, a new legal benchmark to evaluate how well AI models can identify when user queries lack important information needed for accurate legal advice. The study finds that current AI…
Evaluating RAG for French immigration law: a benchmark and baseline study
The paper presents a new benchmark for evaluating AI models used in French immigration law, focusing on permit-type recommendation, required document retrieval, and legal citation coverage. The study compares a…
LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context
Researchers investigated the impact of contextual framing, such as role-play prefixes, on overrefusal rates in small, deployable language models used as legal assistants. They found that authority-style prefixes…
