Abstract Context When conducting a Systematic Literature Review (SLR), researchers usually face the challenge of designing a search strategy that appropriately balances result quality and review effort. Using digital library (or database) searches or snowballing alone may not be enough to achieve high-quality results. On the other hand, using both digital library searches and snowballing together may increase the overall review effort. Objective The goal of this research is to propose and evaluate hybrid search strategies that selectively combine database searches with snowballing. Method We propose four hybrid search strategies combining database searches in digital libraries with iterative, parallel, or sequential backward and forward snowballing. We simulated the strategies over three existing SLRs in SE that adopted both database searches and snowballing. We compared the outcome of digital library searches, snowballing, and hybrid strategies using precision, recall, and F-measure to investigate the performance of each strategy. Results Our results show that, for the analyzed SLRs, combining database searches from the Scopus digital library with parallel or sequential snowballing achieved the most appropriate balance of precision and recall. Conclusion We put forward that, depending on the goals of the SLR and the available resources, using a hybrid search strategy involving a representative digital library and parallel or sequential snowballing tends to represent an appropriate alternative to be used when searching for evidence in SLRs.
On the Performance of Hybrid Search Strategies for Systematic Literature Reviews in Software Engineering
Érica Mourão,J. F. Pimentel,Leonardo Gresta Paulino Murta,Marcos Kalinowski,E. Mendes,C. Wohlin
Published 2020 in Information and Software Technology
ABSTRACT
PUBLICATION RECORD
- Publication year
2020
- Venue
Information and Software Technology
- Publication date
2020-04-21
- Fields of study
Computer Science
- Identifiers
- External record
- Source metadata
Semantic Scholar
CITATION MAP
EXTRACTION MAP
CLAIMS
CONCEPTS
- backward snowballing
Snowballing from the reference lists of seed or retrieved studies.
- database search
A study retrieval step that queries a digital library or database directly.
Aliases: database searches
- forward snowballing
Snowballing from papers that cite seed or retrieved studies.
- hybrid search strategy
A search approach that combines database searches with one or more snowballing steps.
Aliases: hybrid strategies
- iterative snowballing
Snowballing repeated over multiple rounds until no additional studies are found.
- parallel snowballing
A search variant that applies backward and forward snowballing in parallel.
- precision
The fraction of retrieved studies that are relevant.
- recall
The fraction of relevant studies that are retrieved.
- representative digital library
A selected digital library used to stand in for a broader source of studies.
- scopus digital library
The Scopus database used as the representative digital library in the evaluation.
Aliases: Scopus
- sequential snowballing
A search variant that applies backward and forward snowballing in sequence.
- snowballing
A study identification technique that follows references or citations from known studies.
- systematic literature review
A structured review process for identifying and synthesizing relevant research studies.
Aliases: SLR
REFERENCES
Showing 1-27 of 27 references · Page 1 of 1