Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Towards Agentic Agent-based Models: Feasibility, Performance, and Statistical Model Checking

Source

arXiv

arxiv.org/abs/2607.17948v1

Read the source

Source

Plain-Language Brief

Researchers explore the feasibility, performance, and reliability of using large language models (LLMs) in Agent-based Models (ABMs), specifically with Mesa ABM models. They introduce a hybrid population to test LLM-driven decisions within ABM simulations, and report preliminary experiments showing that larger LLMs perform better than smaller ones.

What the Document Says

  • The research focuses on Agent-based Models (ABMs) and the potential of using Large Language Models (LLMs) in these models.
  • The study is conducted on Mesa ABM models, a popular Python library for ABMs.
  • An extension to the classical Schelling segregation model is made with a hybrid population, where ordinary agents use symbolic rules, and one agent uses an LLM through tool calls.
  • The LLM-enabled agent classifies neighbors based on natural-language descriptions and counters of similar/different neighbors.
  • Preliminary experiments were conducted with locally served LLMs of varying sizes.
  • Smaller models sometimes fail semantic classification or become operationally unusable due to repeated tool-call generation.
  • Larger tested models pass these preliminary checks.

Key Concepts Mentioned

  • Agent-based Models (ABMs)
  • Large Language Models (LLMs)
  • Mesa ABM models
  • Schelling segregation model
  • statistical model checking
  • tool calls

Claims Made by the Document

  • Using LLM-driven decisions in ABM simulations may affect reliability, computational cost, and behavior.
  • Smaller LLMs may fail semantic classification or become operationally unusable during repeated tool-call generation.
  • Larger tested LLMs pass preliminary checks.

Evidence Used

  • Preliminary experiments with locally served LLMs of different sizes.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Agent-based models (ABMs) rely on simple, explicit and reproducible rules for individual decision making, while complex collective behavior emerges from interactions among agents. Recent advances in large language models (LLMs) make it tempting to replace, enrich, or perturb these rules with LLM-based agentic capabilities. However, this raises a methodological question: how does introducing LLM-driven decisions affect the reliability, computational cost, and behavior of ABM simulations? We investigate this for Mesa ABM models, a popular Python library for ABMs, analyzed by statistical model checking. Building on Mesa's integration with the statistical model checker MultiVeStA, we extend the classical Schelling segregation model with a hybrid population: ordinary agents classify neighbors using the standard symbolic rule, while one agent delegates this task to an LLM through tool calls. The LLM-enabled agent receives natural-language descriptions of neighboring agents and invokes tools that increment counters of similar/different neighbors; these counters determine its happiness according to the original Schelling dynamics. This provides a minimal but controlled setting where the semantic, operational, and computational behavior of LLM-based decisions can be studied inside an otherwise standard ABM. We report preliminary experiments with locally served LLMs of different sizes, showing that smaller models may fail simple semantic classification experiments or become operationally unusable during repeated tool-call generation, while larger tested models pass these preliminary checks. We discuss how statistical model checking can estimate classical ABM observables and quantify the impact of introducing agentic LLM components into simulation models.