Threats to Training: A Survey of Poisoning Attacks and Defenses on Machine Learning Systems

Machine learning (ML) has been universally adopted for automated decisions in a variety of fields, including recognition and classification applications, recommendation systems, natural language processing, and so on. However, in light of high expenses on training data and computing resources, recent years have witnessed a rapid increase in outsourced ML training, either partially or completely, which provides vulnerabilities for adversaries to exploit. A prime threat in training phase is called poisoning attack, where adversaries strive to subvert the behavior of machine learning systems by poisoning training data or other means of interference. Although a growing number of relevant studies have been proposed, the research among poisoning attack is still overly scattered, with each paper focusing on a particular task in a specific domain. In this survey, we summarize and categorize existing attack methods and corresponding defenses, as well as demonstrate compelling application scenarios, thus providing a unified framework to analyze poisoning attacks. Besides, we also discuss the main limitations of current works, along with the corresponding future directions to facilitate further researches. Our ultimate motivation is to provide a comprehensive and self-contained survey of this growing field of research and lay the foundation for a more standardized approach to reproducible studies.

• Publication year

  2022

• Venue

  ACM Computing Surveys

• Publication date

  2022-05-25

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/3538707
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Understanding the Limits of Unsupervised Domain Adaptation via Data Poisoning

  2021cited by this paper
• Federated vs. Centralized Machine Learning under Privacy-elastic Users: A Comparative Analysis

  2020cited by this paper
• Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

  2020cited by this paper
• Poisoning attacks on cyber attack detectors for industrial control systems

  2020cited by this paper
• Effective and Efficient Data Poisoning in Semi-Supervised Learning

  2020cited by this paper
• RC-SSFL: Towards Robust and Communication-efficient Semi-supervised Federated Learning System

  2020cited by this paper
• How Robust are Randomized Smoothing based Defenses to Data Poisoning?

  2020cited by this paper
• Practical Poisoning Attacks on Neural Networks

  2020influential reference
• Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff

  2020cited by this paper
• Deep Poisoning: Towards Robust Image Data Sharing against Visual Disclosure

  2020cited by this paper
• Bait and Switch: Online Training Data Poisoning of Autonomous Driving Systems

  2020cited by this paper
• Min-Max Optimization without Gradients: Convergence and Applications to Black-Box Evasion and Poisoning Attacks

  2020cited by this paper
• Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder

  2020cited by this paper
• Adversarial Attacks on Deep Learning Models of Computer Vision: A Survey

  2020cited by this paper
• Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

  2020influential reference
• Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics

  2020cited by this paper
• Intrinsic Certified Robustness of Bagging against Data Poisoning Attacks

  2020influential reference
• Adversarial Examples on Object Recognition

  2020cited by this paper
• Data Poisoning Attacks Against Federated Learning Systems

  2020cited by this paper
• You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion

  2020influential reference
• Deep Partition Aggregation: Provable Defense against General Poisoning Attacks

  2020cited by this paper
• Weight Poisoning Attacks on Pretrained Models

  2020cited by this paper
• MetaPoison: Practical General-purpose Clean-label Data Poisoning

  2020influential reference
• Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning

  2020influential reference
• FR-Train: A mutual information-based approach to fair and robust training

  2020influential reference
• Influence Function based Data Poisoning Attacks to Top-N Recommender Systems

  2020influential reference
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing

  2020influential reference
• Learning to Detect Malicious Clients for Robust Federated Learning

  2020influential reference
• Online Data Poisoning Attacks

  2020cited by this paper
• Humpty Dumpty: Controlling Word Meanings via Corpus Poisoning

  2020cited by this paper
• Poisoning Attack in Federated Learning using Generative Adversarial Nets

  2019cited by this paper
• Robust Learning from Untrusted Sources

  2019cited by this paper
• Data Poisoning against Differentially-Private Learners: Attacks and Defenses

  2019cited by this paper
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets

  2019influential reference
• Data Poisoning Attacks on Stochastic Bandits

  2019cited by this paper
• Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder

  2019influential reference
• Data Poisoning Attacks in Multi-Party Learning

  2019cited by this paper
• Poisoning Attacks with Generative Adversarial Nets

  2019cited by this paper
• Adversarial Attacks and Defenses in Images, Graphs and Text: A Review

  2019cited by this paper
• Policy Poisoning in Batch Reinforcement Learning and Control

  2019cited by this paper
• A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning

  2019cited by this paper
• Shielding Collaborative Learning: Mitigating Poisoning Attacks Through Client-Side Detection

  2019cited by this paper
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning

  2019influential reference
• Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks

  2019cited by this paper
• Attack-Resistant Federated Learning with Residual-based Reweighting

  2019cited by this paper
• Understanding Distributed Poisoning Attack in Federated Learning

  2019cited by this paper
• Targeted Poisoning Attacks on Social Recommender Systems

  2019cited by this paper
• Deep k-NN Defense Against Clean-Label Data Poisoning Attacks

  2019influential reference
• Learning to Reweight Examples for Robust Deep Learning

  2018cited by this paper
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates

  2018cited by this paper
• Label Sanitization against Label Flipping Poisoning Attacks

  2018cited by this paper
• Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise

  2018cited by this paper
• The Hidden Vulnerability of Distributed Learning in Byzantium

  2018cited by this paper
• Learning under p-Tampering Attacks

  2018cited by this paper
• Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks

  2018influential reference
• Data Poisoning Attacks on Multi-Task Relationship Learning

  2018cited by this paper
• Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

  2018cited by this paper
• Data Poisoning Attacks in Contextual Bandits

  2018cited by this paper
• The Curse of Concentration in Robust Learning: Evasion and Poisoning Attacks from Concentration of Measure

  2018cited by this paper
• Poisoning Attacks to Graph-Based Recommender Systems

  2018cited by this paper
• Spectral Signatures in Backdoor Attacks

  2018cited by this paper
• Stronger data poisoning attacks break data sanitization defenses

  2018cited by this paper
• Analyzing Federated Learning through an Adversarial Lens

  2018influential reference
• Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

  2018influential reference
• Blockwise p-Tampering Attacks on Cryptographic Primitives, Extractors, and Learners

  2017cited by this paper
• Adversarial Examples: Attacks and Defenses for Deep Learning

  2017cited by this paper
• Efficient Label Contamination Attacks Against Black-Box Learning Models

  2017cited by this paper
• Learning from Noisy Labels with Distillation

  2017cited by this paper
• Understanding Black-box Predictions via Influence Functions

  2017cited by this paper
• Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization

  2017influential reference
• Learning from Noisy Large-Scale Datasets with Minimal Supervision

  2017cited by this paper
• A game-theoretic analysis of label flipping attacks on distributed support vector machines

  2017cited by this paper
• Robust Linear Regression Against Training Data Poisoning

  2017cited by this paper
• Certified Defenses for Data Poisoning Attacks

  2017cited by this paper
• Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent

  2017cited by this paper
• Automated poisoning attacks and defenses in malware detection systems: An adversarial machine learning approach

  2017cited by this paper
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning

  2017cited by this paper
• Generative Poisoning Attack Method Against Neural Networks

  2017cited by this paper
• Distributed Statistical Machine Learning in Adversarial Settings

  2017cited by this paper
• Auror: defending against poisoning attacks in collaborative deep learning systems

  2016cited by this paper
• Communication-Efficient Learning of Deep Networks from Decentralized Data

  2016cited by this paper
• Data Poisoning Attacks on Factorization-Based Collaborative Filtering

  2016cited by this paper
• Machine learning: Trends, perspectives, and prospects

  2015cited by this paper
• Using Machine Teaching to Identify Optimal Training-Set Attacks on Machine Learners

  2015influential reference
• Is Feature Selection Secure against Training Data Poisoning?

  2015cited by this paper
• MLaaS: Machine Learning as a Service

  2015cited by this paper
• Robust Logistic Regression and Classification

  2014cited by this paper
• Classification in the Presence of Label Noise: A Survey

  2014cited by this paper
• Intriguing properties of neural networks

  2013cited by this paper
• Adversarial Label Flips Attack on Support Vector Machines

  2012cited by this paper
• Poisoning Attacks against Support Vector Machines

  2012influential reference
• The security of machine learning

  2010cited by this paper
• Casting out Demons: Sanitizing Training Data for Anomaly Sensors

  2008cited by this paper
• Exploiting Machine Learning to Subvert Your Spam Filter

  2008influential reference
• Can machine learning be secure?

  2006cited by this paper
• A theory of the learnable

  1984cited by this paper

• Robust Federated Learning With Heterogeneous Clients via Classifier Calibration and Alignment

  2026cites this paper
• Exploring knowledge poisoning attacks to retrieval-augmented generation

  2026cites this paper
• FRAME: Comprehensive risk assessment framework for adversarial machine learning threats

  2026cites this paper
• A belief rule-based system for online and centralized collaborative performance assessment of networked physical systems subject to nonideal channels

  2026cites this paper
• PDFL: A Privacy-Enhancing and Robust Poisoning Defense Federated Learning Scheme

  2026cites this paper
• HidAttack: An Effective and Undetectable Model Poisoning Attack to Federated Recommenders

  2025influential citation
• RobustPFL: Robust Personalized Federated Learning

  2025cites this paper
• Document Screenshot Retrievers are Vulnerable to Pixel Poisoning Attacks

  2025cites this paper
• On the Application of a Sparse Data Observers (SDOs) Outlier Detection Algorithm to Mitigate Poisoning Attacks in UltraWideBand (UWB) Line-of-Sight (LOS)/Non-Line-of-Sight (NLOS) Classification

  2025cites this paper
• FedGhost: Data-Free Model Poisoning Enhancement in Federated Learning

  2025cites this paper
• GAD-SISA: A Scalable Defense Against Label Flipping Attack

  2025cites this paper
• Data Poisoning in Deep Learning: A Survey

  2025cites this paper
• Integration of IoT in Healthcare Using Advanced Deep Learning Algorithms: A Research Survey

  2025cites this paper
• Prevention of Data Poisonous Threats on Machine Learning Models in e-Health

  2025cites this paper
• Recent Insights in Responsible AI Development and Deployment in National Defense: A Review of Literature, 2022–2024

  2025cites this paper
• Restricted Boltzmann machine with Sobel filter dense adversarial noise secured layer framework for flower species recognition

  2025cites this paper
• Federated learning for privacy-preserving data analytics in mobile applications

  2025cites this paper
• A Systematic Review on the Practicality of Poisoning Defenses in Federated IoT Systems

  2025cites this paper
• SATversary: Adversarial Attacks on Satellite Fingerprinting

  2025cites this paper
• Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices

  2025cites this paper
• Q-Detection: A Quantum-Classical Hybrid Poisoning Attack Detection Method

  2025cites this paper
• RAG Safety: Exploring Knowledge Poisoning Attacks to Retrieval-Augmented Generation

  2025cites this paper
• Emerging AI threats in cybercrime: a review of zero-day attacks via machine, deep, and federated learning

  2025cites this paper
• Mitigation of Poisoning Attacks in Radio Frequency Fingerprinting (RFF) with Outlier Detection Algorithms

  2025cites this paper
• Practical Framework for Privacy-Preserving and Byzantine-Robust Federated Learning

  2025cites this paper
• FLIT: Federated ledger and intelligence for trust-bootstrapping in the Internet of Medical Things

  2025cites this paper
• Stop Reducing Responsibility in LLM-Powered Multi-Agent Systems to Local Alignment

  2025cites this paper
• Security and Privacy Challenges in AI‐Powered Library Recommender Systems: A Systematic Literature Review

  2025cites this paper
• Decentralized finance (DeFi) for financial institutions: a sentiment analysis and Islamic perspective

  2025cites this paper
• Poisoning Attacks and Defenses Techniques for Federated Learning: A Survey

  2025cites this paper
• Implementation-Aware AI Management Framework for AI-Native Networks

  2025cites this paper
• Uncertainty-Guided Evolutionary Game-Theoretic Client Selection for Federated Intrusion Detection in IoT

  2025cites this paper
• Characterizing Cross-Contamination on Multitask Multimodal Large Language Models

  2025cites this paper
• The role of large language models (LLMs) in enhancing intelligent transportation systems: A survey

  2025cites this paper
• Security Challenges in AI Systems: A Comprehensive Survey for Sustainable AI Development

  2025influential citation
• Robust Methods for Deep-Learning Training

  2025cites this paper
• AutoDetect: designing an autoencoder-based detection method for poisoning attacks on object detection applications in the military domain

  2025cites this paper
• ShalDeepDP: A Survey and Comparison of Data Poisoning in Statistical, Classical Deep Learning and Foundation Models

  2025cites this paper
• Reliable federated learning based on delayed gradient aggregation for intelligent connected vehicles

  2025cites this paper
• Ethical perspectives on deployment of large language model agents in biomedicine: a survey

  2025cites this paper
• SATversary: Adversarial Attacks and Defenses for Satellite Fingerprinting

  2025cites this paper
• Defending Data Poisoning Attack Through Watermarked Friendly Noise Luminosity Activated Dense Layered UNet for Classification of Lung Disease

  2024cites this paper
• Defending Against Data Poisoning Attack in Federated Learning With Non-IID Data

  2024cites this paper
• Verifying in the Dark: Verifiable Machine Unlearning by Using Invisible Backdoor Triggers

  2024cites this paper
• A dilution-based defense method against poisoning attacks on deep learning systems

  2024cites this paper
• The Art of Deception: Robust Backdoor Attack using Dynamic Stacking of Triggers

  2024cites this paper
• The impact of transitive annotation on the training of taxonomic classifiers

  2024cites this paper
• ADMIn: Attacks on Dataset, Model and Input. A Threat Model for AI Based Software

  2024cites this paper
• Locality-Based Action-Poisoning Attack against the Continuous Control of an Autonomous Driving Model

  2024cites this paper
• Machine learning and deep learning for user authentication and authorization in cybersecurity: A state-of-the-art review

  2024cites this paper
• Energy Minimization for Participatory Federated Learning in IoT Analyzed via Game Theory

  2024cites this paper
• Machine learning security and privacy: a review of threats and countermeasures

  2024cites this paper
• Robustness verification of k-nearest neighbors by abstract interpretation

  2024cites this paper
• A Survey on Semantic Communication Networks: Architecture, Security, and Privacy

  2024influential citation
• Securing Artificial Intelligence: Exploring Attack Scenarios and Defense Strategies

  2024influential citation
• A Survey on Machine Unlearning: Techniques and New Emerged Privacy Risks

  2024cites this paper
• Footprints of Data in a Classifier: Understanding the Privacy Risks and Solution Strategies

  2024cites this paper
• Age-of-Information-Aware Federated Learning

  2024cites this paper
• The revolution and vision of explainable AI for Android malware detection and protection

  2024cites this paper
• A Comprehensive Risk Analysis Method for Adversarial Attacks on Biometric Authentication Systems

  2024influential citation
• SoK Paper: Security Concerns in Quantum Machine Learning as a Service

  2024cites this paper
• Unveiling Robustness of Spiking Neural Networks against Data Poisoning Attacks

  2024cites this paper
• Training with Differential Privacy: A Gradient-Preserving Noise Reduction Approach with Provable Security

  2024cites this paper
• The poison of dimensionality

  2024cites this paper
• Machine learning approaches to detect, prevent and mitigate malicious insider threats: State-of-the-art review

  2024cites this paper
• Defusing Dirty Label Backdoor Attack With Differentially Private Data Through Feature Vectorized AlexNet for Face Gender Classification

  2024cites this paper
• Bagging as Defence Mechanism Against Adversarial Attack

  2024influential citation
• Threat Modeling AI/ML With the Attack Tree

  2024cites this paper
• Distributed computing in multi-agent systems: a survey of decentralized machine learning approaches

  2024cites this paper
• Application of Large Language Models in Cybersecurity: A Systematic Literature Review

  2024cites this paper
• Poisoning and Evasion: Deep Learning-Based NIDS under Adversarial Attacks

  2024cites this paper
• Data Poisoning Attacks in the Training Phase of Machine Learning Models: A Review

  2024cites this paper
• A Dual Defense Design Against Data Poisoning Attacks in Deep Learning-Based Recommendation Systems

  2024cites this paper
• Threats, Vulnerabilities, and Controls of Machine Learning Based Systems: A Survey and Taxonomy

  2023cites this paper
• Test-Time Poisoning Attacks Against Test-Time Adaptation Models

  2023cites this paper
• Evaluating Label Flipping Attack in Deep Learning-Based NIDS

  2023cites this paper
• FedSpy: A Secure Collaborative Speech Steganalysis Framework Based on Federated Learning

  2023cites this paper
• You Don't Need Robust Machine Learning to Manage Adversarial Attack Risks

  2023cites this paper
• Research on Data Poisoning Attack against Smart Grid Cyber–Physical System Based on Edge Computing

  2023cites this paper
• Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems

  2023cites this paper
• Toward Robust Recommendation via Real-time Vicinal Defense

  2023cites this paper
• SecK2 – A novel machine learning algorithm for detecting data poisoning attacks

  2023cites this paper
• A Framework for Robustness Evaluation in AI-Based Malware Detectors

  2023cites this paper
• Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities

  2023cites this paper
• Data Poisoning Attacks Against Multimodal Encoders

  2022cites this paper
• Incompatibility Clustering as a Defense Against Backdoor Poisoning Attacks

  2021cites this paper
• Exploring Adversarial Attacks And Defenses On Machine Learning

  year unknowncites this paper
• Towards Safety in Multi-agent Reinforcement Learning through Security and Privacy by Design

  year unknowncites this paper
• PPoison: A Pluggable Poisoning attack against distributed training of split learning

  year unknowncites this paper