SWIFT—Scalable Clustering for Automated Identification of Rare Cell Populations in Large, High-Dimensional Flow Cytometry Datasets, Part 1: Algorithm Design

We present a model‐based clustering method, SWIFT (Scalable Weighted Iterative Flow‐clustering Technique), for digesting high‐dimensional large‐sized datasets obtained via modern flow cytometry into more compact representations that are well‐suited for further automated or manual analysis. Key attributes of the method include the following: (a) the analysis is conducted in the multidimensional space retaining the semantics of the data, (b) an iterative weighted sampling procedure is utilized to maintain modest computational complexity and to retain discrimination of extremely small subpopulations (hundreds of cells from datasets containing tens of millions), and (c) a splitting and merging procedure is incorporated in the algorithm to preserve distinguishability between biologically distinct populations, while still providing a significant compaction relative to the original data. This article presents a detailed algorithmic description of SWIFT, outlining the application‐driven motivations for the different design choices, a discussion of computational complexity of the different steps, and results obtained with SWIFT for synthetic data and relatively simple experimental data that allow validation of the desirable attributes. A companion paper (Part 2) highlights the use of SWIFT, in combination with additional computational tools, for more challenging biological problems. © 2014 The Authors. Published by Wiley Periodicals Inc.

• Publication year

  2014

• Venue

  Cytometry Part A

• Publication date

  2014-02-14

• Fields of study

  Biology, Medicine, Computer Science

• Identifiers
  DOI 10.1002/cyto.a.22446PMID 24677621PMCID 4238829
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• SWIFT—Scalable Clustering for Automated Identification of Rare Cell Populations in Large, High-Dimensional Flow Cytometry Datasets, Part 2: Biological Evaluation

  2014cited by this paper
• Hierarchical Modeling for Rare Event Detection and Cell Subset Alignment across Flow Cytometry Samples

  2013influential reference
• flowPeaks: a fast unsupervised clustering for flow cytometry data via K-means and density peak finding

  2012cited by this paper
• Rapid cell population identification in flow cytometry data

  2011cited by this paper
• Extracting a Cellular Hierarchy from High-dimensional Cytometry Data with SPADE

  2011cited by this paper
• Data reduction for spectral clustering to analyze high throughput flow cytometry data

  2010cited by this paper
• Swift: Scalable weighted iterative sampling for flow cytometry clustering

  2010cited by this paper
• Elucidation of seventeen human peripheral blood B‐cell subsets and quantification of the tetanus response using a density‐based method for the automated identification of cell populations in multidimensional flow cytometry data

  2010influential reference
• The curvHDR method for gating flow cytometry samples

  2010cited by this paper
• Methods for merging Gaussian mixture components

  2010cited by this paper
• Combining Mixture Components for Clustering

  2010cited by this paper
• ImmPort FLOCK: Automated cell population identification in high dimensional flow cytometry data (42.17)

  2009cited by this paper
• Automated high-dimensional flow cytometric data analysis

  2009cited by this paper
• Merging Mixture Components for Cell Population Identification in Flow Cytometry

  2009cited by this paper
• flowClust: a Bioconductor package for automated gating of flow cytometry data

  2009cited by this paper
• Statistical mixture modeling for cell subtype identification in flow cytometry

  2008cited by this paper
• Nine‐color flow cytometry for accurate measurement of T cell subsets and cytokine responses. Part I: Panel design by an empiric approach

  2008cited by this paper
• Automated gating of flow cytometry data via robust model‐based clustering

  2008cited by this paper
• The flow of cytometry into systems biology.

  2007cited by this paper
• Practical Flow Cytometry: Shapiro/Flow Cytometry 4e

  2005cited by this paper
• Standardization of cytokine flow cytometry assays

  2005cited by this paper
• Discriminant Analysis and Statistical Pattern Recognition: McLachlan/Discriminant Analysis & Pattern Recog

  2005cited by this paper
• Practical Flow Cytometry, 4th Edition

  2004cited by this paper
• Seventeen-colour flow cytometry: unravelling the immune system

  2004cited by this paper
• Assessment and pruning of hierarchical model based clustering

  2003cited by this paper
• Model-Based Clustering, Discriminant Analysis, and Density Estimation

  2002cited by this paper
• Unsupervised Learning of Finite Mixture Models

  2002cited by this paper
• Accelerating EM for Large Databases

  2001cited by this paper
• Clustering Massive Datasets With Application in Software Metrics and Tomography

  2001cited by this paper
• Finite Mixture Models

  2000cited by this paper
• Applied smoothing techniques for data analysis : the kernel approach with S-plus illustrations

  1999cited by this paper
• Scaling EM (Expectation Maximization) Clustering to Large Databases

  1998cited by this paper
• A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants

  1998influential reference
• Discriminant Analysis and Statistical Pattern Recognition

  1992cited by this paper
• Automated identification of subpopulations in flow cytometric list mode data using cluster analysis.

  1985cited by this paper
• Practical Flow Cytometry

  1985cited by this paper
• Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper

  1977cited by this paper
• The Hungarian method for the assignment problem

  1955cited by this paper

• Challenges and Adaptations of Model‐Based Clustering for Flow and Mass Cytometry

  2025influential citation
• MuSARCyto: Multi‐Head Self‐Attention‐Based Representation Learning for Unsupervised Clustering of Cytometry Data

  2025cites this paper
• Computationally efficient multi-sample flow cytometry data analysis using Gaussian mixture models

  2025cites this paper
• SWIFT clustering analysis of intracellular cytokine staining flow cytometry data of the HVTN 105 vaccine trial reveals high frequencies of HIV-specific CD4+ T cell responses and associations with humoral responses

  2024cites this paper
• Measurable (Minimal) Residual Disease in Myelodysplastic Neoplasms (MDS): Current State and Perspectives

  2024cites this paper
• Artificial intelligence in clinical multiparameter flow cytometry and mass cytometry-key tools and progress.

  2023cites this paper
• A cell-level discriminative neural network model for diagnosis of blood cancers

  2023cites this paper
• Rarity: discovering rare cell populations from single-cell imaging data

  2023cites this paper
• A cell-level discriminative neural network model for diagnosis of blood cancers

  2023cites this paper
• Life Detection on Icy Moons Using Flow Cytometry and Exogenous Fluorescent Stains

  2023cites this paper
• Rapid Reduction in Staphylococcus aureus in Atopic Dermatitis Subjects Following Dupilumab Treatment.

  2023cites this paper
• Multiomics technologies for comprehensive tumor microenvironment analysis in triple-negative breast cancer under neoadjuvant chemotherapy

  2023cites this paper
• INFLECT: an R-package for cytometry cluster evaluation using marker modality

  2022cites this paper
• Applications of single‐cell multi‐omics sequencing in deep understanding of brain diseases

  2022cites this paper
• Assessment of Automated Flow Cytometry Data Analysis Tools within Cell and Gene Therapy Manufacturing

  2022cites this paper
• UMAP Based Anomaly Detection for Minimal Residual Disease Quantification within Acute Myeloid Leukemia

  2022cites this paper
• A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects

  2022cites this paper
• Measuring inter-cluster similarities with Alpha Shape TRIangulation in loCal Subspaces (ASTRICS) facilitates visualization and clustering of high-dimensional data

  2021cites this paper
• High-throughput single cell data analysis - A tutorial.

  2021cites this paper
• Current trends in flow cytometry automated data analysis software

  2021influential citation
• Tailor: Targeting heavy tails in flow cytometry data with fast, interpretable mixture modeling

  2021cites this paper
• Automated Identification of Cell Populations in Flow Cytometry Data with Transformers

  2021cites this paper
• CYBERTRACK2.0: zero-inflated model-based cell clustering and population tracking method for longitudinal mass cytometry data

  2020cites this paper
• Unique natural killer cell subpopulations are associated with exacerbation risk in chronic obstructive pulmonary disease

  2020cites this paper
• Detection of Rare Objects by Flow Cytometry: Imaging, Cell Sorting, and Deep Learning Approaches

  2020cites this paper
• SwiftReg cluster registration automatically reduces flow cytometry data variability including batch effects

  2020cites this paper
• CytoGLMM: conditional differential analysis for flow and mass cytometry experiments

  2020cites this paper
• Model-Based Clustering and Classification for Data Science: With Applications in R

  2019cites this paper
• High-dimensional Data

  2019cites this paper
• Network data

  2019cites this paper
• Other Topics

  2019cites this paper
• Model-based Classification

  2019cites this paper
• Bibliography

  2019cites this paper
• Introduction

  2019cites this paper
• Semi-supervised Clustering and Classification

  2019cites this paper
• Model-Based Clustering and Classification for Data Science

  2019cites this paper
• Model-based Clustering: Basic Ideas

  2019cites this paper
• Guidelines for the use of flow cytometry and cell sorting in immunological studies (second edition)

  2019cites this paper
• Variable Selection

  2019cites this paper
• Discrete Data Clustering

  2019cites this paper
• Flow Plex—A tool for unbiased comprehensive flow cytometry data analysis

  2019cites this paper
• High-speed automatic characterization of rare events in flow cytometric data

  2019cites this paper
• GMM Interpolation for Blood Cell Cluster Alignment in Childhood Leukaemia

  2019cites this paper
• Dealing with Difficulties

  2019cites this paper
• Model-based Clustering with Covariates

  2019cites this paper
• Automated Flow Cytometric MRD Assessment in Childhood Acute B‐ Lymphoblastic Leukemia Using Supervised Machine Learning

  2019cites this paper
• Non-Gaussian Model-based Clustering

  2019cites this paper
• Comparison of clustering methods for multiparametric cytometry data analysis in order to implement an R/Shiny application

  2019cites this paper
• Preface

  2019cites this paper
• Uncertainty Quantification in Multivariate Mixed Models for Mass Cytometry Data

  2019cites this paper
• Phenotyping Immune Cells in Tumor and Healthy Tissue Using Flow Cytometry Data

  2018cites this paper
• Automatically generate two‐dimensional gating hierarchy from clustered cytometry data

  2018cites this paper
• Application of Machine Learning for Automatic MRD Assessment in Paediatric Acute Myeloid Leukaemia

  2018cites this paper
• Clinically validated machine learning algorithm for detecting residual diseases with multicolor flow cytometry analysis in acute myeloid leukemia and myelodysplastic syndrome

  2018cites this paper
• CytoBinning: Immunological insights from multi-dimensional data

  2018cites this paper
• WGAN Latent Space Embeddings for Blast Identification in Childhood Acute Myeloid Leukaemia

  2018cites this paper
• DAFi: A directed recursive data filtering and clustering approach for improving and interpreting data clustering identification of cell populations from polychromatic flow cytometry data

  2018cites this paper
• Sequential Dirichlet process mixtures of multivariate skew $t$-distributions for model-based clustering of flow cytometry data

  2017cites this paper
• Clustering Single-Cell Expression Data Using Random Forest Graphs

  2017cites this paper
• Toward deterministic and semiautomated SPADE analysis

  2017cites this paper
• Mondrian Processes for Flow Cytometry Analysis

  2017cites this paper
• Similarity measure and domain adaptation in multiple mixture model clustering: An application to image processing

  2017cites this paper
• Guidelines for the use of flow cytometry and cell sorting in immunological studies.

  2017cites this paper
• What is a “unimodal” cell population? Using statistical tests as criteria for unimodality in automated gating and quality control

  2017cites this paper
• Automated Analysis of Flow Cytometry Data to Reduce Inter-Lab Variation in the Detection of Major Histocompatibility Complex Multimer-Binding T Cells

  2017cites this paper
• DAFi: A Directed Recursive Filtering and Clustering Approach to Data-Driven Identification of Cell Populations from Polychromatic Flow Cytometry Data

  2017cites this paper
• Finite Mixture Models in Biostatistics

  2017cites this paper
• Feature-based Comparison of Flow Cytometry Data

  2017cites this paper
• Rare Cells: Focus on Detection and Clinical Relevance

  2017cites this paper
• Clustering of cell populations in flow cytometry data using a combination of Gaussian mixtures

  2016cites this paper
• A Multi-Objective Flow Cytometry Profiling for B-Cell Lymphoma Diagnosis

  2016cites this paper
• Multi‐parametric cytometry from a complex cellular sample: Improvements and limits of manual versus computational‐based interactive analyses

  2016cites this paper
• The end of gating? An introduction to automated analysis of high dimensional cytometry data

  2016cites this paper
• Automated analysis of flow cytometry data: a systematic review of recent methods

  2016cites this paper
• Flow Cytometry based automatic MRD assessment in Acute Lymphoblastic Leukaemia: Longitudinal evaluation of time-specific cell population models

  2016cites this paper
• BayesFlow: latent modeling of flow cytometry cell populations

  2016influential citation
• Computational flow cytometry: helping to make sense of high-dimensional immunology data

  2016cites this paper
• Clinical Outcome Prediction Using Single-Cell Data

  2016cites this paper
• Consistent quantitative gene product expression: #1. Automated identification of regenerating bone marrow cell populations using support vector machines

  2016cites this paper
• FloReMi: Flow density survival regression using minimal feature redundancy

  2016cites this paper
• Modeling of inter‐sample variation in flow cytometric data with the joint clustering and matching procedure

  2016cites this paper
• The Role of Machine Learning in Medical Data Analysis. A Case Study: Flow Cytometry

  2016influential citation
• Clinical Outcome Prediction Using Single-Cell Data.

  2016cites this paper
• Correlation-based iterative clustering methods for time course data: The identification of temporal gene response modules for influenza infection in humans

  2016cites this paper
• immunoClust—An automated analysis pipeline for the identification of immunophenotypic signatures in high‐dimensional cytometric datasets

  2015influential citation
• Integrative analysis of high-dimensional data applied to vaccine research

  2015cites this paper
• Advanced flow cytometric analysis of nanoparticle targeting to rare leukemic stem cells in peripheral human blood in a defined model system

  2015cites this paper
• FlowSOM: Using self‐organizing maps for visualization and interpretation of cytometry data

  2015cites this paper
• Computational prediction of manually gated rare cells in flow cytometry data

  2015cites this paper
• Competitive SWIFT cluster templates enhance detection of aging changes

  2015cites this paper
• Latent modeling of flow cytometry cell populations

  2015influential citation
• High-Dimensional Cytometry Data Algorithmic Tools for Mining

  2015cites this paper
• Algorithmic Tools for Mining High-Dimensional Cytometry Data

  2015cites this paper
• On automated Flow Cytometric analysis for MRD estimation of Acute Lymphoblastic Leukaemia: A comparison among different approaches

  2015cites this paper
• Methods for discovery and characterization of cell subsets in high dimensional mass cytometry data.

  2015cites this paper
• SWIFT—Scalable Clustering for Automated Identification of Rare Cell Populations in Large, High-Dimensional Flow Cytometry Datasets, Part 2: Biological Evaluation

  2014influential citation
• A survey of clustering techniques for big data analysis

  2014cites this paper
• Iterative Subsampling in Solution Path Clustering of Noisy Big Data

  2014cites this paper
• An animated landscape representation of CD4+ T-cell differentiation, variability, and plasticity: Insights into the behavior of populations versus cells

  2014cites this paper
• Towards automation of flow cytometric analysis for quality-assured follow-up assessment to guide curative therapy for acute lymphoblastic leukaemia in children

  2014cites this paper