Employing Multi-Objective Search to Enhance Reactive Test Case Generation and Prioritization for Testing Industrial Cyber-Physical Systems

The test case generation and prioritization of industrial cyber-physical systems face critical challenges, and simulation-based testing is one of the most commonly used techniques for testing these complex systems. However, simulation models of industrial CPSs are usually very complex, and executing the simulations becomes computationally expensive, which often make it infeasible to execute all the test cases. To address these challenges, this paper proposes a multi-objective test generation and prioritization approach for testing industrial CPSs by defining a fitness function with four objectives and designing different crossover and mutation operators. We empirically evaluated our fitness function and designed operators along with five multi-objective search algorithms [e.g., nondominated sorting genetic algorithm (NSGA-II)] using four case studies. The evaluation results demonstrated that NSGA-II achieved significantly better performance than the other algorithms and managed to improve random search for on average 43.80% for each objective and 49.25% for the quality indicator hypervolume.

• Publication year

  2018

• Venue

  IEEE Transactions on Industrial Informatics

• Publication date

  2018-03-01

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1145/3205651.3208212
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Model-Based Testing of Cyber-Physical Systems

  2017cited by this paper
• Search-based test case generation for Cyber-Physical Systems

  2017cited by this paper
• Data-Centered Runtime Verification of Wireless Medical Cyber-Physical System

  2017cited by this paper
• Industrial automation based on cyber-physical systems technologies: Prototype implementations and challenges

  2016cited by this paper
• Search-based test case selection of cyber-physical system product lines for simulation-based validation

  2016influential reference
• Test Case Prioritization of Configurable Cyber-Physical Systems with Weight-Based Search Algorithms

  2016influential reference
• Testing the Untestable - Model Testing of Complex Software-Intensive Systems

  2016influential reference
• Industry 4.0 as a Cyber-Physical System study

  2016cited by this paper
• Sound Test-Suites for Cyber-Physical Systems

  2016cited by this paper
• A Practical Guide to Select Quality Indicators for Assessing Pareto-Based Search Algorithms in Search-Based Software Engineering

  2016cited by this paper
• Cyber-physical systems challenges: a needs analysis for collaborating embedded software systems

  2016cited by this paper
• Testing advanced driver assistance systems using multi-objective search and neural networks

  2016cited by this paper
• Automatic generation of test system instances for configurable cyber-physical systems

  2016influential reference
• Test Set Diameter: Quantifying the Diversity of Sets of Test Cases

  2015cited by this paper
• Software Verification and Validation Laboratory:

  2015influential reference
• FeatureIDE: An extensible framework for feature-oriented software development

  2014cited by this paper
• Automating humanitarian missions with a heterogeneous fleet of vehicles

  2014cited by this paper
• WiP abstract: Conformance testing as falsification for cyber-physical systems

  2014cited by this paper
• Electronic Communications of the EASST Volume 70 ( 2014 ) Proceedings of the 14 th International Workshop on Automated Verification of Critical Systems ( AVoCS 2014 ) Conformance Testing of Cyber-Physical Systems : A Comparative Study

  2014cited by this paper
• Parameter tuning or default values? An empirical investigation in search-based software engineering

  2013influential reference
• Achieving scalable model-based testing through test case diversity

  2013cited by this paper
• Industrial Cyber-Physical Systems - iCyPhy

  2013cited by this paper
• Introduction to Embedded Systems - A Cyber-Physical Systems Approach

  2013cited by this paper
• Conformance testing for cyber-physical systems

  2012cited by this paper
• Virtual prototyping of Cyber-Physical Systems

  2012cited by this paper
• Evolutionary functional black-box testing in an industrial setting

  2012cited by this paper
• A practical guide for using statistical tests to assess randomized algorithms in software engineering

  2011influential reference
• jMetal: A Java framework for multi-objective optimization

  2011cited by this paper
• Model-Based Verification of Embedded Software

  2009cited by this paper
• Model-based Testing of Real-Time Embedded Systems in the Automotive Domain

  2009cited by this paper

• FlowRepair: Search-based automated program repair of CPS controllers modeled in Simulink-Stateflow

  2026cites this paper
• Enhanced Deep Reinforcement Learning for Robustness Falsification of Partially Observable Cyber-Physical Systems

  2026cites this paper
• Multi-Objective Optimization-Based Test Selection Algorithm Considering Test Reliability

  2025cites this paper
• Test case specification techniques and system testing tools in the automotive industry: A review

  2025cites this paper
• An Empirical Evaluation of White-box and Black-box Test Case Prioritization Techniques in CPSs Modeled in Simulink

  2025cites this paper
• Using quantum annealing to generate test cases for cyber-physical systems

  2025cites this paper
• Signal Feature Coverage and Testing for CPS Dataflow Models

  2025cites this paper
• A Multi-Objective Test Scenario Prioritization Method Based on UML Activity Diagram

  2025cites this paper
• NLPLego: Assembling Test Generation for Natural Language Processing Applications

  2024cites this paper
• Safety Verification of Advanced Driver Assistance Systems Using Hybrid Automaton Reachability

  2024cites this paper
• Search-based Automated Program Repair of CPS Controllers Modeled in Simulink-Stateflow

  2024cites this paper
• On the Benefits of Semi-Supervised Test Case Generation for Simulation Models

  2023cites this paper
• Uncertainty-Aware Test Prioritization: Approaches and Empirical Evaluation

  2023cites this paper
• Design a robust quantitative feedback theory controller for cyber-physical systems: ship course control problem

  2023cites this paper
• Test Case Generation and Test Oracle Support for Testing CPSs using Hybrid Models

  2023influential citation
• A Multi-Goal Particle Swarm Optimizer for Test Case Prioritization

  2023cites this paper
• Diversity-based Test Case Prioritization Technique to Improve Faults Detection Rate

  2023cites this paper
• A microservice-based framework for multi-level testing of cyber-physical systems

  2023cites this paper
• Applying and Extending the Delta Debugging Algorithm for Elevator Dispatching Algorithms (Experience Paper)

  2023cites this paper
• On the Benefits of Semi-Supervised Test Case Generation for Cyber-Physical Systems

  2023cites this paper
• Specification-Based Autonomous Driving System Testing

  2023cites this paper
• Automated Misconfiguration Repair of Configurable Cyber-Physical Systems with Search: an Industrial Case Study on Elevator Dispatching Algorithms

  2023cites this paper
• Some Seeds Are Strong: Seeding Strategies for Search-based Test Case Selection

  2022cites this paper
• Cyber-Physical Systems as Key Element to Industry 4.0: Characteristics, Applications and Related Technologies

  2022cites this paper
• What Not to Test (For Cyber-Physical Systems)

  2021cites this paper
• Dual-Network Task Scheduling inCyber–Physical Systems: A Cooptimization Approach

  2021cites this paper
• Alternatives for Testing of Context-Aware Contemporary Software Systems in industrial settings: Results from a Rapid review

  2021cites this paper
• Parallel and Multi-Objective Falsification with Scenic and VerifAI

  2021cites this paper
• Automated Test Cases Prioritization for Self-driving Cars in Virtual Environments

  2021cites this paper
• Model-based test case generation and prioritization: a systematic literature review

  2021cites this paper
• A Systematic Literature Review on Regression Test Case Prioritization

  2021cites this paper
• Survey on test case generation, selection and prioritization for cyber‐physical systems

  2021cites this paper
• Machine learning-based test selection for simulation-based testing of self-driving cars software

  2021influential citation
• Faster Multi-Goal Simulation-Based Testing Using DoLesS (Domination with Least Square Approximation)

  2021cites this paper
• Systematic Literature Review on Test Case Selection and Prioritization: A Tertiary Study

  2021cites this paper
• Single and Multi-objective Test Cases Prioritization for Self-driving Cars in Virtual Environments

  2021influential citation
• A Tool for the Automatic Generation of Test Cases and Oracles for Simulation Models Based on Functional Requirements

  2020cites this paper
• Seeding strategies for multi-objective test case selection: an application on simulation-based testing

  2020cites this paper
• Automated Test Case Generation in Software Testing

  2020cites this paper
• Pareto efficient multi-objective black-box test case selection for simulation-based testing

  2019cites this paper
• Data Management in Industry 4.0: State of the Art and Open Challenges

  2019cites this paper
• TeMA: A Tensorial Memetic Algorithm for Many-Objective Parallel Disassembly Sequence Planning in Product Refurbishment

  2019cites this paper
• Search-Based test case prioritization for simulation-Based testing of cyber-Physical system product lines

  2019cites this paper
• A Systematic Literature Review of Test Case Prioritization Using Genetic Algorithms

  2019influential citation
• A CRITICAL ANALYSIS OF SWARM INTELLIGENCE FOR REGRESSION TEST CASE PRIORITIZATION

  year unknowncites this paper