Autonomous Science Bench: Evaluating AI Agents for Autonomous Scientific Discovery

Source facts

Autonomous Science Bench introduces six budgeted design-build-test-learn campaigns grounded in published biology, chemistry, and materials experiments. Agents choose batches of designs, receive lab-service measurements, update their strategy, and produce a predictive model. Success is separated into a discovery gate and a learning gate. In a pilot of eight agent-model configurations and 144 trials, the best configuration succeeded in 67% of trials; the hardest campaign was solved in three of 24 attempts. Version 0.1 includes campaigns, a lab runtime, and a public leaderboard.

Analysis

The two-gate design separates finding promising candidates from learning to rank unseen candidates. That exposes failure modes hidden by a single overall success rate and creates a more realistic test for agents operating under experimental budgets.

Uncertainty/Doubt

• This is an arXiv preprint. The campaign validity, lab runtime, pilot results, and leaderboard were not independently reproduced here.