DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design

Source facts

DRYLAB-Bench evaluates agentic mutation-design capability in 30 hazardous scenarios distilled from biosecurity guidelines, using multi-round tool-augmented interactions. It scores designs against 187,695 empirical wet-lab measurements from ProteinGym and ViroGym through a proposed Biosecurity Risk Score. The abstract reports evaluations of 15 frontier models, including 67.6% of one-shot designs exceeding the average measured variant without tools and a higher pooled risk and top-decile share when biological design tools are added.

Analysis

The benchmark shifts attention from hazardous knowledge possession to tool-mediated design behavior. Its use of existing empirical measurements is intended to provide scalable verification without new wet-lab experimentation, while the results frame tool access as a measurable risk amplifier.

Uncertainty/Doubt

• This is an arXiv preprint. The hazardous scenarios, scoring calibration, model evaluations, and biosecurity interpretation were not independently reproduced here.