OpenMTB-Audit: Exposing Over-Refusal and Clinical Expert Perspectives in LLM-Based Molecular Tumor Board Safety Evaluation

Source facts

The preprint introduces a benchmark of 500 synthetic non-small-cell lung-cancer cases with four safety labels, including Supported, Partially Supported, Unsupported, and Insufficient Information. Across eight model configurations, the abstract reports that models often collapsed the Partially Supported category, producing high aggregate safety scores through over-refusal. The proposed deterministic audit framework separates evidence verification, missing-information detection, safety classification, and abstention.

Analysis

The study highlights that conservative refusal can mask poor clinical calibration. Preserving distinctions between partial support, missing information, and unsupported recommendations may be as important as reducing hallucinations in high-stakes medical workflows.

Uncertainty/Doubt

• This is an arXiv preprint accepted for a future symposium. The synthetic-case design, model configurations, and clinician annotation boundaries were not independently reproduced here.