On the performance of hybrid search strategies for systematic literature reviews in software engineering
Using a hybrid search strategy involving a representative digital library and parallel or sequential snowballing tends to represent an appropriate alternative to be used when searching for evidence in SLRs.
Abstract
Context: When conducting a Systematic Literature Review (SLR), researchers\nusually face the challenge of designing a search strategy that appropriately\nbalances result quality and review effort. Using digital library (or database)\nsearches or snowballing alone may not be enough to achieve high-quality\nresults. On the other hand, using both digital library searches and snowballing\ntogether may increase the overall review effort.\n Objective: The goal of this research is to propose and evaluate hybrid search\nstrategies that selectively combine database searches with snowballing.\n Method: We propose four hybrid search strategies combining database searches\nin digital libraries with iterative, parallel, or sequential backward and\nforward snowballing. We simulated the strategies over three existing SLRs in SE\nthat adopted both database searches and snowballing. We compared the outcome of\ndigital library searches, snowballing, and hybrid strategies using precision,\nrecall, and F-measure to investigate the performance of each strategy.\n Results: Our results show that, for the analyzed SLRs, combining database\nsearches from the Scopus digital library with parallel or sequential\nsnowballing achieved the most appropriate balance of precision and recall.\n Conclusion: We put forward that, depending on the goals of the SLR and the\navailable resources, using a hybrid search strategy involving a representative\ndigital library and parallel or sequential snowballing tends to represent an\nappropriate alternative to be used when searching for evidence in SLRs.\n