Business processes have become ubiquitous in industry today. They form the main ingredient of business process management. The two most prominent standardized languages to model business processes are Web Services Business Process Execution Language 2.0 (BPEL) and Business Process Model and Notation 2.0 (BPMN). Business process engines allow for automatic execution of business processes. There is a plethora of business process engines available, and thus, one has the agony of choice: which process engine fits the demands the best? The lack of objective, reproducible, and ascertained information about the quality of such process engines makes rational choices very difficult. This can lead to baseless and premature decisions that may result in higher long term costs. This work provides an effective and efficient benchmarking solution to reveal the necessary information to allow making rational decisions. The foundation comprises an abstraction layer for process engines that provides a uniform API to interact with any engine similarly and a benchmark language for process engines to represent benchmarks in a concise, self-contained, and interpretable domain-specific language. A benchmark framework for process engines performs benchmarks represented in this language on engines implementing the abstraction layer. The produced benchmark results are visualized and made available for decision makers via a public interactive dashboard. On top of that, the efficient benchmark framework uses virtual machines to improve test isolation and reduce “time to result” by snapshot restoration accepting a management overhead. Based on the gained experience, eight challenges faced in process engine benchmarking are identified, resulting in 21 process engine benchmarking. Results show that this approach is both effective and efficient. Effective because it covers four BPEL-based and another four BPMN-based benchmarks which cover half of the quality characteristics defined by the ISO/IEC 25010 product quality model. Efficient because it fully automates the benchmarking of process engines and can leverage virtualization for an even higher execution efficiency. With this approach, the barrier for creating good benchmarks is significantly lowered. This allows decision makers to consistently evaluate process engines and, thus, makes rational decisions for the corresponding selection possible.
Effective and Efficient Process Engine Evaluation
Published 2017 in Schriften aus der Fakultät Wirtschaftsinformatik und Angewandte Informatik der Otto-Friedrich-Universität Bamberg
ABSTRACT
PUBLICATION RECORD
- Publication year
2017
- Venue
Schriften aus der Fakultät Wirtschaftsinformatik und Angewandte Informatik der Otto-Friedrich-Universität Bamberg
- Publication date
Unknown publication date
- Fields of study
Business, Computer Science
- Identifiers
- External record
- Source metadata
Semantic Scholar
CITATION MAP
EXTRACTION MAP
CLAIMS
- No claims are published for this paper.
CONCEPTS
- No concepts are published for this paper.
REFERENCES
CITED BY
Showing 1-2 of 2 citing papers · Page 1 of 1