Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Collective cooperation without individual fidelity in LLM agents

Source

arXiv

arxiv.org/abs/2606.30454v1

Read the source

Source

Plain-Language Brief

Researchers compared the behavior of Large Language Models (LLMs) to humans in a networked Prisoner's Dilemma experiment. While LLMs could replicate some collective cooperation patterns, they failed to match human individual-level behavior and decision rules.

What the Document Says

  • The study investigates the use of Large Language Models as agents in social system simulations.
  • The research compares LLMs with human participants in a large-scale Prisoner's Dilemma experiment using the same protocol, payoff structure, and network topologies.
  • Nine open-weight LLMs were tested against human data in terms of cooperation dynamics.
  • The selected LLM reproduced macro-level features of cooperation dynamics such as an early decline and later stabilization of cooperation.
  • However, the LLM populations underestimated individual-level heterogeneity and generated conditional cooperation patterns different from those observed in humans.
  • Adding a fraction of random agents improved some aspects of micro-level agreement but did not remove the mismatch in decision rules.
  • The findings suggest a macro--micro dissociation in LLM-based social agents: collective outcomes can appear human-like despite non-human underlying behavioral distributions and mechanisms.
  • Validating LLMs as human surrogates requires comparisons across aggregate dynamics, individual heterogeneity, and context-dependent decision rules, rather than outcome-level agreement alone.

Key Concepts Mentioned

  • Large Language Models (LLMs)
  • Prisoner's Dilemma experiment
  • human participants
  • cooperation dynamics
  • individual-level behavior
  • decision rules

Claims Made by the Document

  • The selected LLM reproduces several macro-level features of cooperation dynamics.
  • LLM populations underestimate individual-level heterogeneity and generate conditional cooperation patterns different from those observed in humans.
  • Adding a fraction of random agents improves some aspects of micro-level agreement but does not remove the mismatch in decision rules.

Evidence Used

  • The study compares LLMs with human data in a large-scale Prisoner's Dilemma experiment.
  • Results show that while LLMs reproduce macro-level cooperation dynamics, they differ from humans in terms of individual behavior and decision rules.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Large language models (LLMs) are increasingly used as agents in simulations of social systems, yet it remains unclear when their behavior can be interpreted as a faithful proxy for human decision-making. Here we test LLM agents against a direct empirical benchmark: a large-scale networked Prisoner's Dilemma experiment with human participants. Using the same interaction protocol, payoff structure, and network topologies, we compare nine open-weight LLMs with the human data. The selected model reproduces several macro-level features of cooperation dynamics, including the early decline and later stabilization of cooperation. This aggregate agreement, however, does not extend uniformly to finer levels of behavior. LLM populations underestimate individual-level heterogeneity and generate conditional cooperation patterns that differ from those observed in humans. Adding a fraction of random agents improves some aspects of micro-level agreement, but does not remove the mismatch in decision rules. These findings reveal a macro--micro dissociation in LLM-based social agents: collective outcomes can appear human-like even when the underlying behavioral distributions and mechanisms are not. They suggest that validating LLM agents as human surrogates requires comparisons across aggregate dynamics, individual heterogeneity, and context-dependent decision rules, rather than outcome-level agreement alone.