Detecting Anomalous Behavior in Cloud Servers by Nested-Arc Hidden SEMI-Markov Model with State Summarization

Anomaly detection for cloud servers is important for detecting zero-day attacks. However, it is very challenging due to the large amount of accumulated data. In this paper, a new mathematical model for modeling dynamic usage behavior and detecting anomalies is proposed. It is constructed using state summarization and a novel nested-arc hidden semi-Markov model (NAHSMM). State summarization is designed to extract usage behavior reflective states from a raw sequence. The NAHSMM is comprised of exterior and interior hidden Markov chains. The exterior controls the propagation of raw sequences of system calls and, conditional on it, the interior one controls the summarized observation process from the transition less usage behavior reflective states. An anomaly detection algorithm is derived by integrating state summarization and NAHSMM. During training the algorithm is assisted by a forensic module to tune the behavioral threshold. Experimental data is collected using IXIA Perfect Storm in conjunction with the commercial security-test hardware platform cyber range. To evaluate the reliability of the proposed model, first, its accuracy and training costs are compared with those of existing machine-learning models and then its scalability and resistance capabilities are tested. The results indicate that this model could be used as a method for detecting anomalies in cloud servers.

• Publication year

  2019

• Venue

  IEEE Transactions on Big Data

• Publication date

  2019-09-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/TBDATA.2017.2736555
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Generating realistic intrusion detection system dataset based on fuzzy qualitative modeling

  2017influential reference
• Distributed denial of service (DDoS) resilience in cloud: Review and conceptual cloud DDoS mitigation framework

  2016cited by this paper
• Windows Based Data Sets for Evaluation of Robustness of Host Based Intrusion Detection Systems (IDS) to Zero-Day and Stealth Attacks

  2016cited by this paper
• Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing

  2016cited by this paper
• User profiling in intrusion detection: A review

  2016cited by this paper
• Energy Big Data Analytics and Security: Challenges and Opportunities

  2016cited by this paper
• Distributed Real-Time Anomaly Detection in Networked Industrial Sensing Systems

  2015cited by this paper
• Integer Data Zero-Watermark Assisted System Calls Abstraction and Normalization for Host Based Anomaly Detection Systems

  2015cited by this paper
• Anomaly detection in big data from UWB radars

  2015cited by this paper
• A trace abstraction approach for host-based anomaly detection

  2015cited by this paper
• Towards reliable data feature retrieval and decision engine in host-based anomaly detection systems

  2015influential reference
• A partitioning approach to scaling anomaly detection in graph streams

  2014cited by this paper
• Developing a high-accuracy cross platform Host-Based Intrusion Detection System capable of reliably detecting zero-day attacks

  2014cited by this paper
• Evaluating Host-Based Anomaly Detection Systems: Application of the Frequency-Based Algorithms to ADFA-LD

  2014cited by this paper
• Statistical wavelet-based anomaly detection in big data with compressive sensing

  2013cited by this paper
• Modeling Oscillation Behavior of Network Traffic by Nested Hidden Markov Model with Variable State-Duration

  2013cited by this paper
• Network Traffic Classification Using Correlation Information

  2013cited by this paper
• A Survey of Cyber Ranges and Testbeds

  2013cited by this paper
• Addressing cloud computing security issues

  2012cited by this paper
• Machine learning - a probabilistic perspective

  2012cited by this paper
• Study on HMM Based Anomaly Intrusion DetectionUsing System Calls

  2012cited by this paper
• Anomaly detection in large-scale data stream networks

  2012cited by this paper
• Low-Rate DDoS Attacks Detection and Traceback by Using New Information Metrics

  2011cited by this paper
• Detecting Intrusions through System Call Sequence and Argument Analysis

  2010cited by this paper
• Host-Based Anomaly Intrusion Detection

  2010cited by this paper
• Cloud Computing: Implementation, Management and Security, J.W. Ritinghouse, J.F. Ransome. CRC Press (2010)

  2010cited by this paper
• Beyond Output Voting: Detecting Compromised Replicas Using HMM-Based Behavioral Distance

  2009cited by this paper
• Flexible Deterministic Packet Marking: An IP Traceback System to Find the Real Source of Attacks

  2009cited by this paper
• A simple and efficient hidden Markov model scheme for host-based anomaly intrusion detection

  2009cited by this paper
• A new intrusion detection method based on Fuzzy HMM

  2008cited by this paper
• Research on Hidden Markov Model for System Call Anomaly Detection

  2007cited by this paper
• System approach to intrusion detection using hidden Markov model

  2006cited by this paper
• A Hybrid Method to Intrusion Detection Systems Using HMM

  2005cited by this paper
• A Reinforcement Learning Approach for Host-Based Intrusion Detection Using Sequences of System Calls

  2005cited by this paper
• An efficient hidden Markov model training scheme for anomaly intrusion detection of server applications based on system calls

  2004cited by this paper
• Efficient anomaly detection by modeling privilege flows using hidden Markov model

  2003cited by this paper
• Hierarchical structure analysis of sport videos using HMMS

  2003cited by this paper
• Hierarchical Hidden Markov Models for Information Extraction

  2003cited by this paper
• A multi-layer model for anomaly intrusion detection using program sequences of system calls

  2003cited by this paper
• Âóùöòòð Óó Öøø¬ Blockin

  2002cited by this paper
• Layered representations for human activity recognition

  2002cited by this paper
• Multiresolution image classification by hierarchical modeling with two-dimensional hidden Markov models

  2000cited by this paper
• Detecting intrusions using system calls: alternative data models

  1999cited by this paper
• The Hierarchical Hidden Markov Model: Analysis and Applications

  1998influential reference
• A Maximum Likelihood Approach to Continuous Speech Recognition

  1983cited by this paper

• VDDFormer: A Variable Dependency Discrepancy-Based Transformer for Multivariate Time Series Anomaly Detection

  2026cites this paper
• Enhancing Cloud Infrastructure Security with GraphSAGE-Based Intrusion Detection Systems

  2025cites this paper
• Anomaly Detection in Multi-Level Model Space

  2025cites this paper
• Intrusion Detection System Using Generative Unique Adversarial Neural Network in cloud Environment

  2024cites this paper
• AERF: Adaptive ensemble random fuzzy algorithm for anomaly detection in cloud computing

  2023cites this paper
• Anomaly Detection of 5G Control Plane Based on Hidden Semi-Markov Model

  2023cites this paper
• Evaluating Word Embedding Feature Extraction Techniques for Host-Based Intrusion Detection Systems

  2023cites this paper
• An innovative malware detection methodology employing the amalgamation of stacked BiLSTM and CNN+LSTM‐based classification networks with the assistance of Mayfly metaheuristic optimization algorithm in cyber‐attack

  2023cites this paper
• ENetRM: ElasticNet Regression Model based malicious cyber-attacks prediction in real-time server

  2022cites this paper
• Improving IoT data availability via feedback- and voting-based anomaly imputation

  2022cites this paper
• Cyber Threats Prediction Model using Advanced Data Science Approaches

  2022influential citation
• Autocorrelation Integrated Gaussian Based Anomaly Detection using Sensory Data in Industrial Manufacturing

  2021cites this paper
• SCADS: A Scalable Approach Using Spark in Cloud for Host-based Intrusion Detection System with System Calls

  2021influential citation
• A Machine Learning-Based Intrusion Detection System for Securing Remote Desktop Connections to Electronic Flight Bag Servers

  2021cites this paper
• Multi-Parallel Adaptive Grasshopper Optimization Technique for Detecting Anonymous Attacks in Wireless Networks

  2021cites this paper
• Deep Learning-based Anomaly Detection in Cyber-physical Systems

  2021cites this paper
• Minimax Detection (MAD) for Computer Security: A Dynamic Program Characterization

  2021cites this paper
• Secure Continuous-Variable Quantum Key Distribution with Machine Learning

  2021cites this paper
• Comparison of Anomaly Detection Accuracy of Host-based Intrusion Detection Systems based on Different Machine Learning Algorithms

  2020cites this paper
• 2019 Index IEEE Transactions on Big Data Vol. 5

  2020cites this paper
• A Unified Host-based Intrusion Detection Framework using Spark in Cloud

  2020cites this paper
• A Systematic Review of Defensive and Offensive Cybersecurity with Machine Learning

  2020cites this paper
• Hidden-Markov-model-based calibration-attack recognition for continuous-variable quantum key distribution

  2020cites this paper
• FGMC-HADS: Fuzzy Gaussian mixture-based correntropy models for detecting zero-day attacks from linux systems

  2020cites this paper
• Anomaly Detection in Smart Home Operation From User Behaviors and Home Conditions

  2020cites this paper
• Deep Learning-Based Anomaly Detection in Cyber-Physical Systems: Progress and Opportunities

  2020cites this paper
• A method for testing distributed anomaly detectors

  2019cites this paper
• Scalable processing methods for host-based intrusion detection systems

  2019influential citation
• Machine Learning Based Intrusion Detection for Virtualized Infrastructures

  2018cites this paper
• A New Threat Intelligence Scheme for Safeguarding Industry 4.0 Systems

  2018influential citation
• A systematic review on intrusion detection based on the Hidden Markov Model

  2018cites this paper
• Host-Based Intrusion Detection System with System Calls

  2018influential citation
• Bayesian Rules Based Optimal Defense Strategies for Clustered WSNs

  2018cites this paper
• Designing Anomaly Detection System for Cloud Servers by Frequency Domain Features of System Call Identifiers and Machine Learning

  2017cites this paper
• Using LSTM Encoder-Decoder Algorithm for Detecting Anomalous ADS-B Messages

  2017cites this paper