Language-model groups overstate consensus when replaying human deliberation on a reasoning task

Source facts

The preprint replays 100 held-out human Wason groups with matched LLM agent groups, using belief-anchored agents and shared scoring code. It reports that consensus estimates depend on participation and final-state definitions, that agent groups were substantially more consensual under several comparisons, and that simulated consensus did not track collective accuracy; reasoning-mode groups could agree nearly unanimously on incorrect answers.

Analysis

The result is a warning against treating agreement among simulated agents as a direct proxy for human collective cognition or correctness. Participation asymmetry and answer-quality measurement need to be controlled before consensus is interpreted.

Uncertainty/Doubt

• This is a single-author arXiv preprint; the task, scoring choices, and generalizability beyond the reported Wason setting were not independently verified here.