Interactive Elicitation of Resilience Scenarios Based on Hazard Analysis Techniques

Context. Microservice-based architectures are expected to be resilient. Problem. In practice, the elicitation of resilience requirements and the quantitative evaluation of whether the system meets these requirements is not systematic or not even conducted. Objective. We explore (1) the usage of the scenario-based Architecture Trade-Off Analysis Method (ATAM) and established hazard analysis techniques, i.e., Fault Trees and Control Hazard and Operability Study (CHAZOP), for interactive resilience requirement elicitation and (2) resilience testing through chaos experiments for architecture assessment and improvement. Method. In an industrial setting, we design a structured ATAM-based workshop, including the system's stakeholders, to elicit resilience requirements. To complement the workshop, we develop RESIRIO-a semi-automated, chatbot-assisted, and CHAZOP-based approach-for elicitation. We evaluate RESIRIO through a user study. The requirements from both sources are specified using the ATAM scenario template. We use and extend Chaos Toolkit to transform and automate two scenarios. We quantitatively evaluate these scenarios and suggest resilience improvements based on resilience patterns. Result. We identify 12 resilience scenarios in the workshop. We share lessons learned from the study. In particular, our work provides evidence that an ATAM-based workshop is intuitive to stakeholders in an industrial setting and that stakeholders can quickly learn to use RESIRIO in order to successfully obtain new scenarios. Conclusion. Our approach helps requirements and quality engineers in interactive resilience requirements elicitation.