Balancing the criticisms: Validating multi-agent models of social systems

Abstract Using multi-agent models to study social systems has attracted criticisms because of the challenges involved in their validation. Common criticisms that we have encountered are described, and for each one we attempt to give a balanced perspective of the criticism. A model of intra-state conflict is used to help demonstrate these points. We conclude that multi-agent models for social systems are most useful when (1) the connection between micro-behaviors and macro-behaviors are not well-understood and (2) when data collection from the real-world system is prohibitively expensive in terms of time or money or if it puts human lives at risk.

• Publication year

  2008

• Venue

  Simulation modelling practice and theory

• Publication date

  2008-02-01

• Fields of study

  Sociology, Computer Science, Political Science

• Identifiers
  DOI 10.1016/j.simpat.2007.11.011
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Transparency International Corruption Perceptions Index 2007

  2009cited by this paper
• Developing Theory Through Simulation Methods

  2007cited by this paper
• Agent Interactions in Construct : An Empirical Validation using Calibrated Grounding

  2007cited by this paper
• BioWar: scalable agent-based model of bioattacks

  2006cited by this paper
• Sociology and Simulation: Statistical and Qualitative Cross‐Validation1

  2005cited by this paper
• A Hybrid Micro-Simulator for Determining the Effects of Governmental Control Policies on Transport Chains

  2004cited by this paper
• Implications for Model Validation of Multiresolution, Multiperspective Modeling {MRMPM} and Exploratory Analysis

  2004cited by this paper
• Going Beyond the Data: Empirical Validation Leading to Grounded Theory

  2004cited by this paper
• Computational Laboratories for Organization Science: Questions, Validity and Docking

  2003cited by this paper
• Modeling civil violence: An agent-based computational approach

  2002cited by this paper
• Agent Based Social Simulation: A Computer Science View

  2002cited by this paper
• Computational organizational science and organizational engineering

  2002cited by this paper
• Multi-agent simulation for crisis management

  2002cited by this paper
• Multi Agent Based Simulation: Beyond Social Simulation

  2000cited by this paper
• Jeffrey Pfeffer and Gerald R. Salancik: The External Control of Organizations : A Resource Dependence Perspective

  2000cited by this paper
• Validation and verification of simulation models

  1999influential reference
• A Trajectory for Validating Computational Emulation Models of Organizations

  1999cited by this paper
• Validation of models: statistical techniques and data availability

  1999cited by this paper
• Validation in Simulation: Various Positions in the Philosophy of Science

  1998cited by this paper
• A Structural Theory of Social Influence: List of Tables and Figures

  1998cited by this paper
• Advancing the art of simulation in the social sciences

  1997cited by this paper
• Models of Bounded Rationality: Empirically Grounded Economic Reason

  1997cited by this paper
• The validity of computational models in organization science: From model realism to purpose of the model

  1995cited by this paper
• On modeling processes

  1993cited by this paper
• A Theory of Group Stability

  1991cited by this paper
• Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis

  1990cited by this paper
• PREDICTING WITH NETWORKS: NONPARAMETRIC MULTIPLE REGRESSION ANALYSIS OF DYADIC DATA *

  1988cited by this paper
• Ethnic Groups in Conflict

  1988cited by this paper
• Ethnic Groups in Conflict.

  1986cited by this paper
• Simulation Modeling and Analysis

  1982influential reference
• A methodology for cost-risk analysis in the statistical validation of simulation models

  1981cited by this paper
• The External Control of Organizations: A Resource Dependence Perspective.

  1979cited by this paper
• Inequality and Heterogeneity: A Primitive Theory of Social Structure

  1978cited by this paper
• Economics for social decisions : elements of cost-benefit analysis

  1973cited by this paper
• Essence of Decision: Explaining the Cuban Missile Crisis

  1972cited by this paper
• A Theory of Social Comparison Processes

  1954cited by this paper
• AND T

  year unknowncited by this paper

• Agentově orientované modelování revolučních procesů

  2024cites this paper
• Investigation of the Cyber Vulnerabilities of Construction Networks Using an Agent-Based Model

  2024cites this paper
• Evolution of Policy-Driven Ecosystem of Original Innovation Talents

  2024cites this paper
• Toward parallel intelligence: An interdisciplinary solution for complex systems

  2023cites this paper
• Modeling conditions for effective and equitable land use governance in tropical forest frontiers

  2023cites this paper
• Simulation modeling of operation and coordination of agencies involved in post-disaster response and recovery

  2023cites this paper
• Scale decisions and good practices in socio-environmental systems modelling: guidance and documentation during problem scoping and model formulation

  2023cites this paper
• Hybrid simulation model for navigation performance evaluation of the Three Gorges–Gezhouba Dams under novel regulations

  2022cites this paper
• Multi-agent simulation model updating and forecasting for the evaluation of COVID-19 transmission

  2022cites this paper
• Policy and behavioral response to shock events: An agent-based model of the effectiveness and equity of policy design features

  2022cites this paper
• Effects of disclosing inspection scores of health facilities

  2021cites this paper
• Interorganizational Collaboration in Innovation Networks: An Agent Based Model for Responsible Research and Innovation in Additive Manufacturing

  2021cites this paper
• The Theory-Practice Gap in the Evaluation of Agent-Based Social Simulations

  2021cites this paper
• Towards a framework building for social systems modelling

  2020influential citation
• A multi-agent model of changes in urban safety livability

  2020cites this paper
• Bridging the gaps: using agent-based modeling to reconcile data and theory in computational communication science

  2019cites this paper
• agentenbasierten Modellierung eines schweizerischen Waldenergieholzmarktes

  2019cites this paper
• Evaluating Theories of Collaborative Cognition Using the Hawkes Process and a Large Naturalistic Data Set

  2019cites this paper
• Simulating Complex Social-Behavioral Systems

  2019cites this paper
• Creating a bias in inspection data: Exploring the medium- to long-term effects of data-driven risk-based regulation

  2019cites this paper
• Empirical validation of agent-based models

  2018cites this paper
• Step 1 : Identification of methods and techniques . Step 2 : Identification of simulations ’ properties poten

  2018cites this paper
• A Framework for Optimizing Simulation Model Validation & Verification

  2018cites this paper
• Agent-Based Modelling and Simulation of Product Development Teams

  2018cites this paper
• Causal Analysis of an Agent-Based Model of Human Behaviour

  2017cites this paper
• An analysis of a forward capacity market with long-term contracts

  2017cites this paper
• Learning by failure vs learning by habits: Entrepreneurial learning micro-strategies as determinants of the emergence of co-located entrepreneurial networks

  2017cites this paper
• Agent based modelling as a decision support system for shadow accounting

  2017cites this paper
• Regional Innovation Systems: An Agent-Based Laboratory for Policy Advice

  2017cites this paper
• A Framework for Simulation Validation & Verification Method Selection

  2017cites this paper
• Small-World Networks of corruption

  2016cites this paper
• Modelling socio-ecological systems with MAIA: A biogas infrastructure simulation

  2016cites this paper
• Organizational learning with forgetting: Reconsidering the exploration–exploitation tradeoff

  2016cites this paper
• Exploring vulnerability-induced attack frequencies through agent-based modelling

  2016cites this paper
• Understanding Cyber-risk by Investigating the Behaviour of Defender and Threat Agent Organisations: Why a Complex Adaptive Systems Perspective Contributes to Further Understanding Cyber-risk

  2016cites this paper
• The Vulnerability Ecosystem: Exploring vulnerability discovery and the resulting cyberattacks through agent-based modelling

  2016influential citation
• Security of supply during the energy transition: The role of capacity mechanisms

  2016influential citation
• Online social contagion modeling through the dynamics of Integrate-and-Fire neurons

  2015cites this paper
• Artificial Financial Markets: Historical and Interdisciplinary Perspective

  2015cites this paper
• Agent-Based Modeling and Organization Studies: A critical realist perspective

  2015cites this paper
• The philosophy of computational social science

  2015cites this paper
• Interactions between carbon and power markets in transition

  2015cites this paper
• Agent Based Modeling, Statistics of

  2015cites this paper
• Results of an Agent-Based Market Simulation for Transferable Development Rights (TDR) in Switzerland

  2015cites this paper
• Computational Organizational Theory

  2015cites this paper
• Model Validation and Scenario Analysis

  2015cites this paper
• Computational sociocultural models used for forecasting | 269 10 Computational sociocultural models used for forecasting 1

  2015cites this paper
• Lessons from a comprehensive validation of an agent based-model: The experience of the Pampas Model of Argentinean agricultural systems

  2014cites this paper
• Identifying key parameters and trends in civil violence: a sub-regional, agent-based simulation approach using GIS

  2014influential citation
• The role of languages for modeling and simulating continuous-time multi-level models in demography

  2014cites this paper
• Macroscopic Observation of Large-Scale Multi-agent Systems

  2014cites this paper
• essons from a comprehensive validation of an agent based-model : he experience of the Pampas Model of Argentinean gricultural systems

  2014cites this paper
• A simple geometric validation approach to assess the basic behaviour of space- and time- distributed models of epidemic spread - an example using the Ontario rabies model.

  2014cites this paper
• Enhancing Recycling of Construction Materials: An Agent Based Model with Empirically Based Decision Parameters

  2014cites this paper
• Modeling Reactive Systems Using Cooperating Adaptive Devices

  2014cites this paper
• The complex problem of food safety : Applying agent-based modeling to the policy process

  2014influential citation
• Simulation of a Swiss wood fuel and roundwood market: an explorative study in agent-based modeling

  2014cites this paper
• Rethinking Validation: Efficient Search of the Space of Parameters for an Agent-Based Model

  2014influential citation
• Relationship Strength and Network Form: An Agent-Based Simulation of Interaction in a Business Network

  2014cites this paper
• Parallel Computing in Geography

  2014cites this paper
• The Contribution of Social Simulation in the Advancement of Marketing Issues and Challenges

  2013cites this paper
• Analyzing vulnerabilities between SCADA system and SUC due to interdependencies

  2013cites this paper
• Making the Soldier Decisive on Future Battlefields

  2013cites this paper
• Evaluation des performances des systèmes multi-agents

  2013cites this paper
• Modelling affect regulation dynamics (MARDy) : a computational simulation of affect change

  2013cites this paper
• Agent-Based Modeling and Simulation Validation by Scenario Analysis

  2013cites this paper
• Using a Hybrid Model for Investigating Residential Segregation: An Empirical and Simulation-based Study

  2013cites this paper
• Modélisation des flux routiers de marchandises, de la génération de la demande par une approche multi-agents à l’affectation temporelle

  2013influential citation
• A Computational Model of Public Support for Insurgency and Terrorism: A Prototype for More-General Social-Science Modeling

  2013cites this paper
• Interdisciplinary production of knowledge with participation of stakeholders: A case study of a collaborative project on climate variability, human decisions and agricultural ecosystems in the Argentine Pampas

  2013cites this paper
• A policymaker’s puzzle, or how to cross the boundary from agent-based model to land-use policymaking?

  2013cites this paper
• Modélographie multi-agents de la simulation des interactions sociétés-environnement et de l’émergence des villes

  2013cites this paper
• Towards computational models of cultural dynamics based on the grounding model of cultural transmission

  2012cites this paper
• Early Results from Metamorphic Testing of Epidemiological Models

  2012cites this paper
• Using Case Data to Ensure 'Real World' Input Validation within Fuzzy Set Theory Models

  2012cites this paper
• Evaluation of Distance-Based and Cordon-Based Urban Freight Road Pricing in E-Commerce Environment with Multiagent Model

  2012cites this paper
• Critical perspectives on historical collapse

  2012cites this paper
• A Hybrid Sensitivity Analysis Approach for Agent-based Disease Spread Models

  2012cites this paper
• Business Network Simulation: Combining Research Cases and Agent-Based Models in a Robust Methodology

  2012cites this paper
• Agent-based modelling of transactive memory systems and knowledge processes in agile versus traditional software development teams

  2012cites this paper
• A logic of argumentation for specification and verification of abstract argumentation frameworks

  2012cites this paper
• Modelling the effects of social networks on activity and travel behaviour

  2012cites this paper
• Enhancing the Understanding of Corruption through System Dynamics Modelling: A Case Study Analysis of Pakistan

  2012cites this paper
• Implementation and Evaluation of Hybrid routing protocol for wired / wireless mutually complementary communication protocol

  2011cites this paper
• Agent-based Simulation of Non-linear Interactions in Business Networks

  2011cites this paper
• Observation macroscopique et émergence dans les SMA de très grande taille

  2011influential citation
• Macroscopic Observation of Multiagent Systems February 17 , 2011

  2011cites this paper
• Agent-based modelling of housing choice in the EASEL regeneration district

  2011cites this paper
• Multi-agent model approach to evaluate distance-based urban freight road pricing

  2011cites this paper
• An Agent Operationalization Approach for Context Specific Agent-Based Modeling

  2011influential citation
• A Framework for ICT-Supported Coordination in Crisis Response

  2010cites this paper
• Why Use Agent-Based Models To Explore Social Issues? The Case Of Intimate Partner Violence and Social Support Systems

  2010cites this paper
• A Proposition on Applying Agent-based Model for Analyzing Logistics System

  2010cites this paper
• Material flow dynamics in the automotive industry in Germany an agent based approach

  2010cites this paper
• Validating of complex agent-based models of social activities and travel behaviour

  2010cites this paper
• Theory Development through Simulation: Extending coordination theory in crisis response

  2009cites this paper
• Multi-Agent Modeling to Understand the Impact of Stakeholders' Decisions in a Residential Land Development Project in Southern Alberta

  2009cites this paper
• Artificial markets: A review and assessment of a new venue for innovation research

  2009cites this paper
• Simulación computacional de grupos de investigación en la Universidad Estatal a Distancia de Costa Rica

  2009cites this paper
• Alternative ways of verification and validation of computational models: A case of replication in the innovation networks.

  2009influential citation