Alert Characterization by Non-expert Users in a Cybersecurity Virtual Environment: A Usability Study.

Although cybersecurity is a domain where data analysis and training are considered of the highest importance, few virtual environments for cybersecurity are specifically developed, while they are used efficiently in other domains to tackle these issues. By taking into account cyber analysts' practices and tasks, we have proposed the 3D Cyber Common Operational Picture model (3D Cyber-COP), that aims at mediating analysts' activities into a Collaborative Virtual Environment (CVE), in which users can perform alert analysis scenarios. In this article, we present a usability study we have performed with non-expert users. We have proposed three virtual environments (a graph-based, an office-based, and the coupling of the two previous ones) in which users should perform a simplified alert analysis scenario based on the WannaCry ransomware. In these environments, users must switch between three views (alert, cyber and physical ones) which all contain different kinds of data sources. These data have to be used to perform the investigations and to determine if alerts are due to malicious activities or if they are caused by false positives. We have had 30 users, with no prior knowledge in cybersecurity. They have performed very well at the cybersecurity task and they have managed to interact and navigate easily. SUS usability scores were above 70 for the three environments and users have shown a preference towards the coupled environment, which was considered more practical and useful.