Spartan: A Comprehensive Tool for Understanding Uncertainty in Simulations of Biological Systems

Integrating computer simulation with conventional wet-lab research has proven to have much potential in furthering the understanding of biological systems. Success requires the relationship between simulation and the real-world system to be established: substantial aspects of the biological system are typically unknown, and the abstract nature of simulation can complicate interpretation of in silico results in terms of the biology. Here we present spartan (Simulation Parameter Analysis R Toolkit ApplicatioN), a package of statistical techniques specifically designed to help researchers understand this relationship and provide novel biological insight. The tools comprising spartan help identify which simulation results can be attributed to the dynamics of the modelled biological system, rather than artefacts of biological uncertainty or parametrisation, or simulation stochasticity. Statistical analyses reveal the influence that pathways and components have on simulation behaviour, offering valuable biological insight into aspects of the system under study. We demonstrate the power of spartan in providing critical insight into aspects of lymphoid tissue development in the small intestine through simulation. Spartan is released under a GPLv2 license, implemented within the open source R statistical environment, and freely available from both the Comprehensive R Archive Network (CRAN) and http://www.cs.york.ac.uk/spartan. The techniques within the package can be applied to traditional ordinary or partial differential equation simulations as well as agent-based implementations. Manuals, comprehensive tutorials, and example simulation data upon which spartan can be applied are available from the website.

• Publication year

  2013

• Venue

  PLoS Comput. Biol.

• Publication date

  2013-02-01

• Fields of study

  Biology, Medicine, Computer Science

• Identifiers
  DOI 10.1371/journal.pcbi.1002916PMID 23468606PMCID 3585389
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Differential RET Signaling Pathways Drive Development of the Enteric Lymphoid and Nervous Systems

  2012cited by this paper
• Techniques for grounding agent-based simulations in the real domain: a case study in experimental autoimmune encephalomyelitis

  2012cited by this paper
• Wiley Series in Probability and Statistics

  2012cited by this paper
• Pairing experimentation and computational modeling to understand the role of tissue inducer cells in the development of lymphoid organs

  2012cited by this paper
• Systems biology in immunology: a computational modeling perspective.

  2011cited by this paper
• The CoSMoS Process, Version 0.1: A Process for the Modelling and Simulation of Complex Systems

  2010cited by this paper
• Synergy between Individual TNF-Dependent Functions Determines Granuloma Performance for Controlling Mycobacterium tuberculosis Infection1

  2009cited by this paper
• Uncertainty and Sensitivity Analysis for Models of Complex Systems

  2008influential reference
• Computational methods in transport : verification and validation

  2008cited by this paper
• Mathematical and computational approaches can complement experimental studies of host–pathogen interactions

  2008cited by this paper
• A methodology for performing global uncertainty and sensitivity analysis in systems biology.

  2008influential reference
• Development of secondary lymphoid organs.

  2008cited by this paper
• Wiley Series in Probability and Mathematical Statistics

  2008cited by this paper
• State Dependent Parameter metamodelling and sensitivity analysis

  2007cited by this paper
• Key Role of Local Regulation in Chemosensing Revealed by a New Molecular Interaction-Based Modeling Method

  2006cited by this paper
• Random balance designs for the estimation of first order global sensitivity indices

  2006cited by this paper
• Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models

  2004cited by this paper
• Toward rigorous comprehension of biological complexity: modeling, execution, and visualization of thymic T-cell maturation.

  2003cited by this paper
• Overlapping Roles of CXCL13, Interleukin 7 Receptor α, and CCR7 Ligands in Lymph Node Development

  2003cited by this paper
• IMMSIM, a flexible model for in machina experiments on immune system responses

  2002cited by this paper
• A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong

  2000influential reference
• An alternative way to compute Fourier amplitude sensitivity test (FAST)

  1998cited by this paper

• A model of tuberculosis progression using CompuCell3D

  2026cites this paper
• An agent-based modelling approach to investigate the impact of gender on tuberculosis transmission in Uganda

  2026cites this paper
• Bench to Bedside Modeling of mRNA Encoding IgG Using a Multiscale Mechanistic Pharmacokinetic‐Toxicokinetic (PK‐TK) Model: A Case Study With Anti‐Claudin 18.2

  2025cites this paper
• Investigating the impacts of alcohol outlet zoning policy on alcohol consumption and access to non-alcoholic services: A spatial agent-based simulation.

  2025cites this paper
• The pros and cons of the Preston–Biggins egg shape model: A reconsideration case based on mathematical modeling and simulation

  2025cites this paper
• In silico study of the solubility of fullerene derivatives in chlorobenzene

  2025cites this paper
• Multispecies minimal physiologically based pharmacokinetic-pharmacodynamic model of antisense oligonucleotides for central nervous system disorders.

  2025cites this paper
• Impact of interventions to reduce nosocomial transmission of SARS-CoV-2 in English NHS Trusts: a computational modelling study

  2024cites this paper
• Simulations probe the role of space in the interplay between drug-sensitive and drug-resistant cancer cells

  2024cites this paper
• Histology-guided mathematical model of tumor oxygenation: sensitivity analysis of physical and computational parameters

  2024cites this paper
• Three Types of Isocoumarins with Unusual Carbon Skeletons from Artemisia dubia var. subdigitata and Their Antihepatoma Activity

  2024cites this paper
• Decomposing Variance Decomposition for Stochastic Models: Application to a Proof-Of-Concept Human Migration Agent-Based Model

  2024cites this paper
• The effectiveness and efficiency of asymptomatic SARS-CoV-2 testing strategies for patient and healthcare workers within acute NHS hospitals during an omicron-like period

  2024cites this paper
• Potential and Challenges of Assurance Cases for Simulation Validation

  2024cites this paper
• A quantitative systems pharmacology (QSP) platform for preclinical to clinical translation of in-vivo CRISPR-Cas therapy

  2024cites this paper
• A Quality Diversity Study in EvoDevo Processes for Engineering Design

  2024cites this paper
• Uncertainty Quantification for the Stochastic Modeling of Influenza Propagation: How Many Simulation Runs is Enough

  2024cites this paper
• Implications of reducing antibiotic treatment duration for antimicrobial resistance in hospital settings: A modelling study and meta-analysis

  2023cites this paper
• Trustworthy agent-based simulation: the case for domain-specific modelling languages

  2023cites this paper
• A study of error diversity in robotic swarms for task partitioning in foraging tasks

  2023cites this paper
• Impact of testing on Clostridioides difficile infection in hospitals across Europe: A mathematical model.

  2023cites this paper
• TrackSOM: Mapping immune response dynamics through clustering of time‐course cytometry data

  2022influential citation
• Mathematical modelling of activation-induced heterogeneity in TNF, IL6, NOS2, and IL1β expression reveals cell state transitions underpinning macrophage responses to LPS

  2022cites this paper
• Quantitative Systems Pharmacology Modeling Framework of Autophagy in Tuberculosis: Application to Adjunctive Metformin Host-Directed Therapy

  2022cites this paper
• How do classroom-turnover times depend on lecture-hall size?

  2022cites this paper
• Using single-cell cytometry to illustrate integrated multi-perspective evaluation of clustering algorithms using Pareto fronts

  2021cites this paper
• Neutrophil Dynamics Affect Mycobacterium tuberculosis Granuloma Outcomes and Dissemination

  2021cites this paper
• Simulating CXCR5 Dynamics in Complex Tissue Microenvironments

  2021cites this paper
• Minimal Physiologically Based Pharmacokinetic-Pharmacodynamic (mPBPK-PD) Model of N-Acetylgalactosamine–Conjugated Small Interfering RNA Disposition and Gene Silencing in Preclinical Species and Humans

  2021influential citation
• A mathematical model to study the impact of intra-tumour heterogeneity on anti-tumour CD8+ T cell immune response.

  2021cites this paper
• TrackSOM: mapping immune response dynamics through sequential clustering of time- and disease-course single-cell cytometry data

  2021cites this paper
• Agent-based modelling of the social-ecological nature of poverty traps in southwestern Madagascar

  2021cites this paper
• On the Macroeconomic Effect of Extortion: An Agent-Based Approach

  2021cites this paper
• The impact of testing and infection prevention and control strategies on within-hospital transmission dynamics of COVID-19 in English hospitals

  2020cites this paper
• Using Emulation to Engineer and Understand Simulations of Biological Systems

  2020influential citation
• LYMESIM 2.0: An Updated Simulation of Blacklegged Tick (Acari: Ixodidae) Population Dynamics and Enzootic Transmission of Borrelia burgdorferi (Spirochaetales: Spirochaetaceae)

  2020cites this paper
• Systems biology predicts that fibrosis in tuberculous granulomas may arise through macrophage-to-myofibroblast transformation

  2020cites this paper
• Spatial structure impacts adaptive therapy by shaping intra-tumoral competition

  2020cites this paper
• B cell zone reticular cell microenvironments shape CXCL13 gradient formation

  2020cites this paper
• A Hybrid Model of Cartilage Regeneration Capturing the Interactions Between Cellular Dynamics and Porosity

  2020cites this paper
• Models and Machines: How Deep Learning Will Take Clinical Pharmacology to the Next Level

  2019cites this paper
• Global sensitivity analysis of biological multi-scale models.

  2019cites this paper
• Targeting Cellular DNA Damage Responses: Predicting in vivo treatment responses using an in vitro-calibrated agent-based mathematical model

  2019cites this paper
• Assessing Algorithm Parameter Importance Using Global Sensitivity Analysis

  2019cites this paper
• Targeting Cellular DNA Damage Responses in Cancer: An In Vitro-Calibrated Agent-Based Model Simulating Monolayer and Spheroid Treatment Responses to ATR-Inhibiting Drugs

  2019cites this paper
• Immune-inspired fault diagnosis for robot swarms

  2019cites this paper
• Application of Modeling Approaches to Explore Vaccine Adjuvant Mode-of-Action

  2019cites this paper
• Uncertainty and Sensitivity Analyses Methods for Agent-Based Mathematical Models: An Introductory Review

  2019influential citation
• Playing with Patterns

  2019cites this paper
• IL-10 Impairs Local Immune Response in Lung Granulomas and Lymph Nodes during Early Mycobacterium tuberculosis Infection

  2019cites this paper
• A hybrid cellular automaton model of cartilage regeneration capturing the interactions between cellular dynamics and scaffold porosity

  2018cites this paper
• Agent-Based Modelling to Assess Community Food Security and Sustainable Livelihoods

  2018influential citation
• Macrophage Transactivation for Chemokine Production Identified as a Negative Regulator of Granulomatous Inflammation Using Agent-Based Modeling

  2018cites this paper
• How to Relate Models to Reality? An Epistemological Framework for the Validation and Verification of Computational Models

  2018cites this paper
• Strategies for calibrating models of biology

  2018cites this paper
• Adaptive Online Fault Diagnosis in Autonomous Robot Swarms

  2018cites this paper
• Insights into the dynamics of harmful algal blooms in a tropical estuary through an integrated hydrodynamic-Pyrodinium-shellfish model.

  2018cites this paper
• Significance of multi-site calibration for agent-based transmission models

  2018cites this paper
• A Hormone-Inspired Arbitration System For Self Identifying Abilities Amongst A Heterogeneous Robot Swarm

  2018cites this paper
• Lessons learned on development and application of agent-based models of complex dynamical systems

  2017cites this paper
• Model-Driven Experimentation: A New Approach to Understand Mechanisms of Tertiary Lymphoid Tissue Formation, Function, and Therapeutic Resolution

  2017cites this paper
• Forecasting disturbance effects on wildlife: tolerance does not mitigate effects of increased recreation on wildlands

  2017cites this paper
• Using a multi-objective artificial immune system approach for biodiversity conservation

  2017cites this paper
• Fault diagnosis in robot swarms: An adaptive online behaviour characterisation approach

  2017cites this paper
• Assessing the importance of artificial nest-sites in the population dynamics of endangered Northern Aplomado Falcons Falco femoralis septentrionalis in South Texas using stochastic simulation models

  2017influential citation
• Investigating IKK dynamics in the NF-κB signalling pathway using X-Machines

  2017cites this paper
• Evolving test environments to identify faults in swarm robotics algorithms

  2017influential citation
• The spatial distribution and dynamics of CXCL13 in lymphoid tissues

  2017influential citation
• Building confidence in quantitative systems pharmacology models: An engineer's guide to exploring the rationale in model design and development

  2017cites this paper
• Extending and Applying Spartan to Perform Temporal Sensitivity Analyses for Predicting Changes in Influential Biological Pathways in Computational Models

  2017influential citation
• Agent-Based Modeling of Mitochondria Links Sub-Cellular Dynamics to Cellular Homeostasis and Heterogeneity

  2017cites this paper
• ASPASIA: A toolkit for evaluating the effects of biological interventions on SBML model behaviour

  2017cites this paper
• Characterization and classification of adherent cells in monolayer culture using automated tracking and evolutionary algorithms

  2016cites this paper
• Available Instruments for Analyzing Molecular Dynamics Trajectories

  2016cites this paper
• Wound healing : a multidisciplinary approach : combining mathematical models and biological experiments

  2016cites this paper
• CARDINAL-Vanilla : immune system inspired prioritisation and distribution of security information for industrial networks

  2016cites this paper
• Leukocyte Motility Models Assessed through Simulation and Multi-objective Optimization-Based Model Selection

  2016cites this paper
• Imaging Immunity in Lymph Nodes: Past, Present and Future.

  2016cites this paper
• Correction: Determining Disease Intervention Strategies Using Spatially Resolved Simulations

  2015cites this paper
• Action horizon : on the controllability of complex systems, moving towards management for energy systems

  2015cites this paper
• An integrated molecular cell biology and agent-based simulation approach to dissecting microRNA regulatory networks

  2015influential citation
• Modelling Hospital Acquired Clostridium difficile Infections And Its Transmission In Acute Hospital Settings

  2015cites this paper
• A Multi-objective Optimization Approach Associated to Climate Change Analysis to Improve Systematic Conservation Planning

  2015cites this paper
• Autonomic business processes

  2015cites this paper
• Agent‐Based Modeling in Systems Pharmacology

  2015cites this paper
• Using Multi-Objective Artificial Immune Systems to Find Core Collections Based on Molecular Markers

  2015cites this paper
• Bio-inspired Practicalities: Collective Behaviour using Passive Neighbourhood Sensing

  2015cites this paper
• Principled simulation of cell proliferation dynamics using the CoSMoS approach

  2015cites this paper
• Using argument notation to engineer biological simulations with increased confidence

  2015influential citation
• mtk: A General-Purpose and Extensible R Environment for Uncertainty and Sensitivity Analyses of Numerical Experiments

  2015cites this paper
• Filling gaps in simulation of complex systems: the background and motivation for CoSMoS

  2015cites this paper
• An agent-based model of the IL-1 stimulated Nuclear Factor-kappa B signalling pathway

  2014cites this paper
• Novel Approaches to the Visualization and Quantification of Biological Simulations by Emulating Experimental Techniques

  2014influential citation
• Modelling biological behaviours with the unified modelling language: an immunological case study and critique

  2014cites this paper
• Utilising a simulation platform to understand the effect of domain model assumptions

  2014cites this paper
• Easing Parameter Sensitivity Analysis of Netlogo Simulations Using SPARTAN

  2014cites this paper
• Applying spartan to Understand Parameter Uncertainty in Simulations

  2014cites this paper
• Tuneable resolution as a systems biology approach for multi-scale, multi-compartment computational models

  2014cites this paper
• language : an immunological case study and critique Modelling biological behaviours with the unified modelling

  2014cites this paper
• Biological Data Analysis: Error and Uncertainty

  2013cites this paper