Resilience of Cyber-Physical Systems: an Experimental Appraisal of Quantitative Measures

Cyber-Physical Systems (CPSs) interconnect the physical world with digital computers and networks in order to automate production and distribution processes. Nowadays, most CPSs do not work in isolation, but their digital part is connected to the Internet in order to enable remote monitoring, control and configuration. Such a connection may offer entry-points enabling attackers to gain control silently and exploit access to the physical world at the right time to cause service disruption and possibly damage to the surrounding environment. Prevention and monitoring measures can reduce the risk brought by cyber attacks, but the residual risk can still be unacceptably high in critical infrastructures or services. Resilience - i.e., the ability of a system to withstand adverse events while maintaining an acceptable functionality - is therefore a key property for such systems. In our research, we seek a model-free, quantitative, and general-purpose evaluation methodology to extract resilience indexes from, e.g., system logs and process data. While a number of resilience metrics have already been put forward, little experimental evidence is available when it comes to the cyber security of CPSs. By using the model of a real wastewater treatment plant, and simulating attacks that tamper with a critical feedback control loop, we provide a comparison between four resilience indexes selected through a thorough literature review involving over 40 papers. Our results show that the selected indexes differ in terms of behavior and sensitivity with respect to specific attacks, but they can all summarize and extract meaningful information from bulky system logs. Our evaluation includes an approach for extracting performance indicators from observed variables which does not require knowledge of system dynamics; and a discussion about combining resilience indexes into a single system-wide measure is included. ¹¹ The authors wish to thank Leonardo S.p.A. for its financial support. The research herein presented is partially supported by project NEFERIS awarded by the Italian Ministry of Defense to Leonardo S.p.A. in partnership with the University of Genoa. This work received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No 830892 for project SPARTA.