Structural consistency between a system model and its implementation: a design science study in industry

During the development of complex systems, several different development artifacts are created and maintained. It is important to gain insight into the degree to which these artifacts are consistent, but this is challenging, especially in complex industrial settings. In this study, we aim to help engineers in their consistency management efforts by creating consistency checks between a system model and the corresponding code. To that end, we performed a design science study in which we develop a consistency checking tool and integrate it into an existing industrial system and software engineering setting. We evaluated the developed tool through a case study in which we measure the consistency before and after its introduction and evaluate the experiences of engineers using it. Our results show that the introduction of lightweight consistency checks into the continuous integration pipeline is beneficial for consistency management in the studied setting. Moreover, we discuss the practical challenges of introducing consistency checks in an industrial setting and find that the majority is of a nontechnical nature.