A Survey on Evaluating the Quality of Autonomic Internet of Things Applications

The rapid evolution of the Internet of Things (IoT) facilitates the development of IoT applications in domains such as manufacturing, smart cities, retail, agriculture, etc. Such IoT applications collect data, analyze, and extract insightful information to enable decision-making and actuation. There is an unprecedented growth of IoT applications that automate decision-making and actuation without requiring human intervention, which we term autonomic IoT applications. The increasing scale of such applications necessitates holistic measurement and evaluation of application quality. Existing literature has evaluated quality from an end-user perspective, which may be unsuitable when dealing with the complexity of modern IoT applications, especially when they are autonomic. In this paper, we refer to IoT application quality as the aggregate quantitative value of various IoT quality metrics measured at each stage of the autonomic IoT application life cycle. We present an in-depth survey of current state-of-the-art techniques and approaches for evaluating quality of IoT applications. In particular, we survey various definitions to identify the factors that contribute to understanding and evaluating quality in IoT. Furthermore, we present open issues and identify future research directions towards realizing fine-grained quality evaluation of IoT applications. We envision that the identified research directions will, in turn, enable real-time diagnostics of IoT applications and make them quality-aware. This survey can serve as the basis for designing and developing modern, resilient quality-aware autonomic IoT applications.