Applying and Researching DevOps: A Tertiary Study

DevOps is an emerging software development methodology, that differs from more traditional approaches due to the closer involvement of the customer and the adoption of "continuous-*" (e.g., integration, deployment, delivery, etc.) practices. The vast research on DevOps (including numerous secondary studies) published in a short timeframe, and the diversity of the authors' research backgrounds (e.g., from a Dev or an Ops perspective), has inevitably produced a long list of investigated topics, which use inconsistent terminology. The goal of this study is to analyze literature reviews on DevOps with respect to: (a) the research topics in DevOps; (b) the terms that are mapped to each topic; and (c) the consistency of terminology. To achieve this goal, we have performed a tertiary study, i.e., a systematic mapping study that uses as primary studies "Systematic Literature Reviews" and "Mapping Studies". For Data Extraction, Analysis, and Synthesis (DEAS) we propose a novel approach relying on thematic analysis, statistical analysis, and meta-analysis. The results unveiled 7 core topics on DevOps research, out of which DevOps features and DevOps practices are dominant ones. Additionally, as expected various terminology ambiguities have been identified, most between features as practices, as well as, between challenges faced before adopting DevOps and while applying DevOps. The main contribution of this study is the disambiguation of the mapping of terms to topics. Along this process we highlight both inconsistencies-attempting to resolves ambiguities, as well as topics and terms with high levels of consistency; aiding researchers and practitioners.