Requirements monitoring frameworks: A systematic review.

Analyze the characteristics and application areas of monitoring approaches proposed in different domains.Systematically identify frameworks supporting requirements monitoring.Analyze to what extent the monitoring frameworks support requirements monitoring in SoS. ContextSoftware systems today often interoperate with each other, thus forming a system of systems (SoS). Due to the scale, complexity, and heterogeneity of SoS, determining compliance with their requirements is challenging, despite the range of existing monitoring approaches. The fragmented research landscape and the diversity of existing approaches, however, make it hard to understand and analyze existing research regarding its suitability for SoS. ObjectiveThe aims of this paper are thus to systematically identify, describe, and classify existing approaches for requirements-based monitoring of software systems at runtime. Specifically, we (i) analyze the characteristics and application areas of monitoring approaches proposed in different domains, we (ii) systematically identify frameworks supporting requirements monitoring, and finally (iii) analyze their support for requirements monitoring in SoS. MethodWe performed a systematic literature review (SLR) to identify existing monitoring approaches and to classify their key characteristics and application areas. Based on this analysis we selected requirements monitoring frameworks, following a definition by Robinson, and analyzed them regarding their support for requirements monitoring in SoS. ResultsWe identified 330 publications, which we used to produce a comprehensive overview of the landscape of requirements monitoring approaches. We analyzed these publications regarding their support for Robinson's requirements monitoring layers, resulting in 37 identified frameworks. We investigated how well these frameworks support requirements monitoring in SoS. ConclusionsWe conclude that most existing approaches are restricted to certain kinds of checks, particular types of events and data, and mostly also limited to one particular architectural style and technology. This lack of flexibility makes their application in an SoS context difficult. Also, systematic and automated variability management is still missing. Regarding their evaluation, many existing frameworks focus on measuring the performance overhead, while only few frameworks have been assessed in cases studies with real-world systems.