Search Engine Overlaps: Do they agree or disagree?

Context: Secondary studies, such as systematic literature reviews and mapping studies, are an essential element of the evidence-based paradigm. A critical part of the review process is the identification of all relevant research. As such, any researcher intending to conduct a secondary review should be aware of the strengths and weakness of the search engines available. Objectives: Analyse the overlap between search engine results for software engineering studies. Methods: Three independent studies were conducted to evaluate the overlap between multiple search engines for different search areas. Results: The findings indicate that very little overlap was found between the search engines. Conclusions of the study: To complete a systematic review, researchers must use multiple search terms and search engines. The lack of overlap might also be caused by inconsistent keyword selection amongst authors.