A Survey on Data-driven Network Intrusion Detection

Data-driven network intrusion detection (NID) has a tendency towards minority attack classes compared to normal traffic. Many datasets are collected in simulated environments rather than real-world networks. These challenges undermine the performance of intrusion detection machine learning models by fitting machine learning models to unrepresentative “sandbox” datasets. This survey presents a taxonomy with eight main challenges and explores common datasets from 1999 to 2020. Trends are analyzed on the challenges in the past decade and future directions are proposed on expanding NID into cloud-based environments, devising scalable models for large network data, and creating labeled datasets collected in real-world networks.

• Publication year

  2021

• Venue

  ACM Computing Surveys

• Publication date

  2021-10-07

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1145/3472753
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• An Explainable Machine Learning-based Network Intrusion Detection System for Enabling Generalisability in Securing IoT Networks

  2021cited by this paper
• Explaining Network Intrusion Detection System Using Explainable AI Framework

  2021cited by this paper
• Deep Reinforcement Learning based Intrusion Detection System for Cloud Infrastructure

  2020cited by this paper
• Network Intrusion Detection Based on Supervised Adversarial Variational Auto-Encoder With Regularization

  2020cited by this paper
• Ensemble-Based Online Machine Learning Algorithms for Network Intrusion Detection Systems Using Streaming Data

  2020cited by this paper
• 1D CNN based network intrusion detection with normalization on imbalanced data

  2020influential reference
• The Detection of Network Intrusion Based on Improved Adaboost Algorithm

  2020cited by this paper
• Network Intrusion Detection Combined Hybrid Sampling With Deep Hierarchical Network

  2020cited by this paper
• Adversarial Attacks for Image Segmentation on Multiple Lightweight Models

  2020cited by this paper
• BAT: Deep Learning Methods on Network Intrusion Detection Using NSL-KDD Dataset

  2020cited by this paper
• Robust adaptive multivariate Hotelling's T2 control chart based on kernel density estimation for intrusion detection system

  2020cited by this paper
• Achieving Explainability of Intrusion Detection System by Hybrid Oracle-Explainer Approach

  2020cited by this paper
• An efficient feature selection based Bayesian and Rough set approach for intrusion detection

  2020cited by this paper
• Evolving deep learning architectures for network intrusion detection using a double PSO metaheuristic

  2020cited by this paper
• Interactive three-dimensional visualization of network intrusion detection data for machine learning

  2020cited by this paper
• Data-Driven Edge Intelligence for Robust Network Anomaly Detection

  2020cited by this paper
• Hypervisor-based cloud intrusion detection through online multivariate statistical change tracking

  2020cited by this paper
• Machine learning-driven intrusion detection for Contiki-NG-based IoT networks exposed to NSL-KDD dataset

  2020cited by this paper
• Network Anomaly Detection inside Consumer Networks—A Hybrid Approach

  2020cited by this paper
• CNN-Based Network Intrusion Detection against Denial-of-Service Attacks

  2020cited by this paper
• An effective convolutional neural network based on SMOTE and Gaussian mixture model for intrusion detection in imbalanced dataset

  2020cited by this paper
• LITNET-2020: An Annotated Real-World Network Flow Dataset for Network Intrusion Detection

  2020cited by this paper
• A Method of Few-Shot Network Intrusion Detection Based on Meta-Learning Framework

  2020cited by this paper
• Semisupervised-Learning-Based Security to Detect and Mitigate Intrusions in IoT Network

  2020cited by this paper
• Conditional Wasserstein generative adversarial network-gradient penalty-based approach to alleviating imbalanced data classification

  2020cited by this paper
• Data Processing and Model Selection for Machine Learning-based Network Intrusion Detection

  2020influential reference
• Fast Binary Network Intrusion Detection based on Matched Filter Optimization

  2020cited by this paper
• Explainable AI in Industry: Practical Challenges and Lessons Learned

  2020cited by this paper
• An Explainable Machine Learning Framework for Intrusion Detection Systems

  2020cited by this paper
• Towards Near-Real-Time Intrusion Detection for IoT Devices using Supervised Learning and Apache Spark

  2020cited by this paper
• HELAD: A novel network anomaly detection model based on heterogeneous ensemble learning

  2020cited by this paper
• Towards a Reliable Comparison and Evaluation of Network Intrusion Detection Systems Based on Machine Learning Approaches

  2020cited by this paper
• Siam-IDS: Handling class imbalance problem in Intrusion Detection Systems using Siamese Neural Network

  2020cited by this paper
• Network Intrusion Detection Based on PSO-Xgboost Model

  2020cited by this paper
• CANTransfer: transfer learning based intrusion detection on a controller area network using convolutional LSTM network

  2020cited by this paper
• An Authorized Access Attack Detection Method for Realtime Intrusion Detection System

  2020cited by this paper
• A New Method of Fuzzy Support Vector Machine Algorithm for Intrusion Detection

  2020cited by this paper
• Hierarchical Intrusion Detection Using Machine Learning and Knowledge Model

  2020cited by this paper
• Adaptive intrusion detection via GA-GOGMM-based pattern learning with fuzzy rough set-based attribute selection

  2020cited by this paper
• Error-Bounded Graph Anomaly Loss for GNNs

  2020cited by this paper
• Robust detection for network intrusion of industrial IoT based on multi-CNN fusion

  2020cited by this paper
• An Adaptive Ensemble Machine Learning Model for Intrusion Detection

  2019cited by this paper
• A Semi-Boosted Nested Model With Sensitivity-Based Weighted Binarization for Multi-Domain Network Intrusion Detection

  2019cited by this paper
• Mimic Learning to Generate a Shareable Network Intrusion Detection Model

  2019cited by this paper
• ANID-SEoKELM: Adaptive network intrusion detection based on selective ensemble of kernel ELMs with random features

  2019cited by this paper
• Intrusion Detection Using Big Data and Deep Learning Techniques

  2019cited by this paper
• DAD-MCNN: DDoS Attack Detection via Multi-channel CNN

  2019cited by this paper
• Semi-Supervised K-Means DDoS Detection Method Using Hybrid Feature Selection Algorithm

  2019cited by this paper
• Wireless Network Intrusion Detection Based on Improved Convolutional Neural Network

  2019cited by this paper
• Intelligent temporal classification and fuzzy rough set-based feature selection algorithm for intrusion detection system in WSNs

  2019cited by this paper
• A Fourier Perspective on Model Robustness in Computer Vision

  2019cited by this paper
• Network Intrusion Detection: Based on Deep Hierarchical Network and Original Flow Data

  2019cited by this paper
• Intrusion detection for cloud computing using neural networks and artificial bee colony optimization algorithm

  2019cited by this paper
• Traffic Generation using Containerization for Machine Learning

  2019cited by this paper
• DeepDDoS: Online DDoS Attack Detection

  2019cited by this paper
• A Novel Multimodal-Sequential Approach Based on Multi-View Features for Network Intrusion Detection

  2019cited by this paper
• PAC-GAN: Packet Generation of Network Traffic using Generative Adversarial Networks

  2019cited by this paper
• Anomaly-Based Network Intrusion Detection Using SVM

  2019cited by this paper
• Investigating Adversarial Attacks against Network Intrusion Detection Systems in SDNs

  2019cited by this paper
• Domain Knowledge Aided Explainable Artificial Intelligence for Intrusion Detection and Response

  2019cited by this paper
• A Discretized Extended Feature Space (DEFS) Model to Improve the Anomaly Detection Performance in Network Intrusion Detection Systems

  2019cited by this paper
• WOTBoost: Weighted Oversampling Technique in Boosting for imbalanced learning

  2019cited by this paper
• Big Data defined: a practical review for neurosurgeons.

  2019cited by this paper
• Toward an Online Network Intrusion Detection System Based on Ensemble Learning

  2019influential reference
• Explaining What a Neural Network has Learned: Toward Transparent Classification

  2019cited by this paper
• PCCN: Parallel Cross Convolutional Neural Network for Abnormal Network Traffic Flows Detection in Multi-Class Imbalanced Network Traffic Flows

  2019cited by this paper
• Research of Intrusion Detection Method Based on IL-FSVM

  2019cited by this paper
• Evaluating Deep Learning Based Network Intrusion Detection System in Adversarial Environment

  2019cited by this paper
• Overcoming the Lack of Labeled Data: Training Intrusion Detection Models Using Transfer Learning

  2019influential reference
• A Novel Online Incremental Learning Intrusion Prevention System

  2019influential reference
• Deep and Machine Learning Approaches for Anomaly-Based Intrusion Detection of Imbalanced Network Traffic

  2019influential reference
• Cyber intrusion detection by combined feature selection algorithm

  2019cited by this paper
• Dimensionality reduction with IG-PCA and ensemble classifier for network intrusion detection

  2019cited by this paper
• NSL-KDD Dataset

  2019cited by this paper
• Generative Adversarial Networks For Launching and Thwarting Adversarial Attacks on Network Intrusion Detection Systems

  2019cited by this paper
• A Survey of Network-based Intrusion Detection Data Sets

  2019cited by this paper
• GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection

  2019cited by this paper
• KDD Cup 99 Data Sets: A Perspective on the Role of Data Sets in Network Intrusion Detection Research

  2019cited by this paper
• Transfer learning for detecting unknown network attacks

  2019cited by this paper
• UGR'16: A new dataset for the evaluation of cyclostationarity-based network IDSs

  2018cited by this paper
• An Intrusion Detection System Using a Deep Neural Network With Gated Recurrent Units

  2018cited by this paper
• A Novel Semi-Supervised Learning Approach for Network Intrusion Detection on Cloud-Based Robotic System

  2018cited by this paper
• Performance evaluation of intrusion detection based on machine learning using Apache Spark

  2018influential reference
• An Adversarial Approach for Explainable AI in Intrusion Detection Systems

  2018cited by this paper
• A novel architecture combined with optimal parameters for back propagation neural networks applied to anomaly network intrusion detection

  2018cited by this paper
• A Deep Learning Approach to Network Intrusion Detection

  2018cited by this paper
• Deep Learning Approach Combining Sparse Autoencoder With SVM for Network Intrusion Detection

  2018influential reference
• Network intrusion detection using equality constrained-optimization-based extreme learning machines

  2018cited by this paper
• Flow-based Network Traffic Generation using Generative Adversarial Networks

  2018influential reference
• Toward Explainable Deep Neural Network Based Anomaly Detection

  2018cited by this paper
• vNIDS: Towards Elastic Security with Safe and Efficient Virtualization of Network Intrusion Detection Systems

  2018cited by this paper
• A Survey of Random Forest Based Methods for Intrusion Detection Systems

  2018influential reference
• Benchmarking datasets for Anomaly-based Network Intrusion Detection: KDD CUP 99 alternatives

  2018cited by this paper
• Evading Botnet Detectors Based on Flows and Random Forest with Adversarial Samples

  2018cited by this paper
• A GA-LR wrapper approach for feature selection in network intrusion detection

  2017cited by this paper
• An incremental intrusion detection system using a new semi‐supervised stream classification method

  2017cited by this paper
• A local density-based approach for outlier detection

  2017cited by this paper
• A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection

  2017cited by this paper
• Generating realistic intrusion detection system dataset based on fuzzy qualitative modeling

  2017cited by this paper
• Incremental k-NN SVM method in intrusion detection

  2017cited by this paper

• Design and evaluation of a robust and explainable intrusion detection framework for 5G/B5G networks

  2026cites this paper
• A Comparative Analysis of Self-Aware Reinforcement Learning Models for Real-Time Intrusion Detection in Fog Networks

  2026cites this paper
• An efficient framework for malicious network traffic detection using optimized deep learning techniques

  2026cites this paper
• A survey on intelligent detection for APT attacks

  2025cites this paper
• Cybersecurity in the AI era: analyzing the impact of machine learning on intrusion detection

  2025cites this paper
• Towards Better-Calibrated ML Models for Reliable Network Intrusion Detection via Calibration-Aware SHAP-Based Feature Selection

  2025cites this paper
• Explainable AI-Based Intrusion Detection Systems for Industry 5.0 and Adversarial XAI: A Systematic Review

  2025cites this paper
• Ensemble Machine Learning for UAV Network Intrusion Detection: Comprehensive Analysis Using the UAV-NIDD Dataset

  2025cites this paper
• A Comparison study of various Wireless Intrusion Detection Systems

  2025cites this paper
• Mapping the Landscape of Generative AI in Network Monitoring and Management

  2025cites this paper
• Vulnerability disclosure through adaptive black-box adversarial attacks in network intrusion detection systems

  2025cites this paper
• Detection of Multi-Stage Attacks Through Attack Pattern Segmentation

  2025cites this paper
• NOCTOWL: Adaptive Tree-Based Model for Network Anomaly Detection Under Delayed and Sampled Label Availability

  2025cites this paper
• Development of K-Neareast Neighbor Based Model for Network Intrusion Detection Using Random Forest for Feature Selection

  2025cites this paper
• Lightweight Service Mesh for Intrusion Detection using KD-CNN in Cloud-Native Environments

  2025cites this paper
• Cyberattacks Classification by Tuning Deep Hyperparameters Using Bayesian Optimization

  2025cites this paper
• A Hybrid Petri Net–AI Architecture for Adaptive and Explainable Cybersecurity in Business Workflows

  2025cites this paper
• Network Intrusion Detection and Defense System Based on Deep Learning

  2025cites this paper
• Intrusion Security Detection in E-Commerce Data Cloud Computing Networks Based on the GA-BP Network Algorithm

  2025influential citation
• Intrusion detection systems in IoT: A detailed review of threat categories, detection strategies, and future technologies

  2025cites this paper
• An Interpretable Network Intrusion Detection Model via Decision Tree Enhanced Deep Attention Network

  2025cites this paper
• Toward Scalable and Sustainable Detection Systems: A Behavioural Taxonomy and Utility-Based Framework for Security Detection in IoT and IIoT

  2025cites this paper
• Microsecond Federated SVD on Grassmann Manifold for Real-time IoT Intrusion Detection

  2025cites this paper
• An efficient data driven framework for intrusion detection in wireless sensor networks using deep learning

  2025cites this paper
• FWA-SVM Network Intrusion Identification Technology for Network Security

  2025cites this paper
• A Survey on Explainable Artificial Intelligence for Internet Traffic Classification and Prediction, and Intrusion Detection

  2025cites this paper
• ENHANCING SECURITY RESILIENCE: DYNAMIC DRIFT DETECTION IN CLOUD-BASED INTRUSION DETECTION USING THE HYBRID MODEL

  2025cites this paper
• A Unified Framework for Hybrid Network Intrusion Detection

  2025cites this paper
• RustiFlow: Bridging the Gap Between Security Research and Practice Using eBPF-Based Network Flow Extraction

  2025cites this paper
• MTRC: A self-supervised network intrusion detection framework based on multiple Transformers enabled data reconstruction with contrastive learning

  2025cites this paper
• Source identification for worm propagation: A graph neural network approach and evaluation on social network and internet datasets

  2025cites this paper
• A network intrusion detection system based on self-supervised learning of traffic differentiation in Internet of Things

  2025cites this paper
• Mathematical Modeling of Communication Data Encryption Protection and Intelligent Identification of Network Intrusion

  2025cites this paper
• Adaptive Network Threat Detection Using Attention-Based LSTM

  2025cites this paper
• Agentic Graph Neural Networks for Wireless Communications and Networking Toward Edge General Intelligence: A Survey

  2025cites this paper
• REAL-IoT: Characterizing GNN Intrusion Detection Robustness under Practical Adversarial Attack

  2025cites this paper
• Vulnerability Disclosure through Adaptive Black-Box Adversarial Attacks on NIDS

  2025cites this paper
• АНАЛІЗ МОДЕЛЕЙ ВИЯВЛЕННЯ ВТОРГНЕНЬ НА ОСНОВІ НЕЙРОМЕРЕЖ У СИСТЕМАХ ІНТЕРНЕТУ РЕЧЕЙ

  2025cites this paper
• Analysis of the Impact of Attacks Against Machine Learning Components of Intrusion Detection Systems

  2025cites this paper
• Construction of VAE-GRU-XGBoost intrusion detection model for network security

  2025cites this paper
• Safety assessment model for DoS attacks detection in wireless communication and network OS environments

  2025cites this paper
• Enhancing Intrusion Detection Systems with representation methods: A comparative study

  2025cites this paper
• DAD: Enhancing Multi-Class DDoS Attack Classification using Data Augmentation with DRCGAN

  2025cites this paper
• A systematic literature review of log-correlation tools for cyberattack detection and prediction in large networks

  2025cites this paper
• Descriptor: Firewall Attack Detections and Extractions (FADE)

  2025cites this paper
• A DDoS attack detection method based on IQR and DFFCNN in SDN

  2025cites this paper
• Toward Efficient Network Traffic Classifications via Multimodal Learning

  2025cites this paper
• A Survey of Data Stream-Based Intrusion Detection Systems

  2025influential citation
• Classification and analysis of network traffic cyber attacks using decision tree compared with K-nearest neighbors algorithm to improved accuracy

  2025cites this paper
• Toward Realistic Adversarial Attacks in IDS: A Novel Feasibility Metric for Transferability

  2025cites this paper
• Enhancing Cyber Attack Detection: A Comparative Analysis of Labeled and Unlabeled Data in Machine Learning Models

  2025cites this paper
• Enhancing IoT Network Intrusion Detection with a New GraphSAGE Embedding Algorithm Using Centrality Measures

  2025cites this paper
• Explainable AI‐Driven Firewall Evaluation: Empowering Cybersecurity Decision‐Making for Optimal Network Defense

  2025cites this paper
• Network Intrusion Monitoring Based on Margin Distance Pruning and RF Algorithm

  2025cites this paper
• Advanced IDS: a comparative study of datasets and machine learning algorithms for network flow-based intrusion detection systems

  2025cites this paper
• A novel classification method based on an online extended belief rule base with a human-in-the-loop strategy

  2025cites this paper
• Enhancing Intrusion Detection and Cloud Security by Integrating Snort with Advanced AI Techniques for Improved Accuracy and Threat Mitigation

  2025cites this paper
• NIDP: Solving Feature Distribution Shifts in Network Intrusion Detection via Neural Pruning

  2025cites this paper
• XAI-based Customer Behavior Analysis for Network Intrusion Detection in Edge Intelligence

  2025cites this paper
• A lightweight IoT intrusion detection method based on two-stage feature selection and Bayesian optimization

  2025cites this paper
• Generative Active Adaptation for Drifting and Imbalanced Network Intrusion Detection

  2025cites this paper
• Data Quality Tools to Enhance a Network Anomaly Detection Benchmark

  2025cites this paper
• BedIDS: An Effective Network Anomaly Detection Method by Fusing Behavior Evolution characteristics

  2024cites this paper
• QPause: Quantum-Resistant Password-Protected Data Outsourcing for Cloud Storage

  2024cites this paper
• HyperDetect: A Real-Time Hyperdimensional Solution for Intrusion Detection in IoT Networks

  2024cites this paper
• Amplification methods to promote the attacks against machine learning-based intrusion detection systems

  2024cites this paper
• Applying Self-supervised Learning to Network Intrusion Detection for Network Flows with Graph Neural Network

  2024cites this paper
• Learn-IDS: Bridging Gaps between Datasets and Learning-Based Network Intrusion Detection

  2024cites this paper
• Evaluating the Impact of Data Preprocessing Techniques on the Performance of Intrusion Detection Systems

  2024cites this paper
• Machine learning based Cyber Attack detection on Internet Traffic

  2024cites this paper
• GDLC: A new Graph Deep Learning framework based on centrality measures for intrusion detection in IoT networks

  2024cites this paper
• A Review of Advancements and Applications of Pre-Trained Language Models in Cybersecurity

  2024cites this paper
• An Ensemble Framework for Network Anomaly Detection Using Isolation Forest and Autoencoders

  2024cites this paper
• GraphWeaver: Billion-Scale Cybersecurity Incident Correlation

  2024cites this paper
• Heterogeneous GNN with Express Edges for Intrusion Detection in Cyber-Physical Systems

  2024cites this paper
• Swarm-intelligence for the modern ICT ecosystems

  2024cites this paper
• Federated PCA on Grassmann Manifold for IoT Anomaly Detection

  2024cites this paper
• ReCDA: Concept Drift Adaptation with Representation Enhancement for Network Intrusion Detection

  2024cites this paper
• Evaluating The Explainability of State-of-the-Art Machine Learning-based IoT Network Intrusion Detection Systems

  2024cites this paper
• Early Network Intrusion Detection Enabled by Attention Mechanisms and RNNs

  2024cites this paper
• Developing a Hybrid Detection Approach to Mitigating Black Hole and Gray Hole Attacks in Mobile Ad Hoc Networks

  2024cites this paper
• A Looping Process for Cyberattack Mitigation

  2024cites this paper
• Adversarial Challenges in Network Intrusion Detection Systems: Research Insights and Future Prospects

  2024cites this paper
• Two-Stage Intrusion Detection Approach with SMOTE-Enhanced Attack Classification

  2024cites this paper
• A Survey for Deep Reinforcement Learning Based Network Intrusion Detection

  2024cites this paper
• Fed-Evolver: An automated evolving approach for federated Intrusion Detection System using adversarial autoencoder in SDN-enabled networks

  2024cites this paper
• Attack stage detection method based on vector reconstruction error autoencoder and explainable artificial intelligence

  2024cites this paper
• High-performance network attack detection in unknown scenarios based on improved vertical model

  2024cites this paper
• A hybrid feature selection and aggregation strategy-based stacking ensemble technique for network intrusion detection

  2024cites this paper
• Internet of vehicles intrusion detection method based on CFS-COA feature selection and spatio-temporal feature extraction

  2024cites this paper
• Machine Learning in Intrusion Detection: An Operational Perspective

  2024cites this paper
• A Comparison Between Classical and Quantum Machine Learning for Mobile App Traffic Classification

  2024cites this paper
• Semi-Supervised Learning Trumps Active Learning in Cold-Starting Netflow-Based AI Network Intrusion Detection Systems on Scarce and Difficult Data

  2024cites this paper
• Optimizing Network Cybersecurity: AI-Powered NLP for Natural Language Command Interpretation

  2024cites this paper
• Improving IDS Performance with XGBoost Hyperparameter Optimization and Real-Time Analysis

  2024cites this paper
• Sin-Cos Fusion-Based African Vulture Optimization Algorithm for Feature Selection in Detecting DDoS Attack in IoT Networks

  2024cites this paper
• Boosting Network Intrusion Detection with Two-Level Ensemble Learning and Knowledge Distillation Approaches

  2024cites this paper
• Evaluating The Explainability of State-of-the-Art Deep Learning-based Network Intrusion Detection Systems

  2024cites this paper
• ExpIDS: A Drift-Adaptable Network Intrusion Detection System with Improved Explainability

  2024cites this paper
• Emergent Deep Learning for Anomaly Detection in Internet of Everything

  2023cites this paper