A novel framework for verification and validation of simulations of autonomous robots

Modeling and simulation (M&S) plays a critical role in both engineering and basic research processes. However, M&S is only truly useful if the model and simulation outputs are accurate. As such, significant research has been undertaken to establish what “accurate” means for simulations and what subsequent level of “trust” should be given to M&S outputs. Trust in M&S outputs is established by verifying and validating the models and simulations. While a wealth of research can be found to define verification and validation (V&V) for traditional M&S, little research has been done to define a methodology for the V&V of simulations of complex, intelligent, and autonomous systems. Specifically, no methodology for V&V of simulations of autonomous robots has been developed to date. This paper presents a brief overview of the current V&V methods in use for traditional simulations. In light of this review, a novel framework for the V&V of simulations for predicting the behaviors of autonomous robots is developed, and this framework is presented in detail. This V&V framework is then applied to the use-case of an autonomous unmanned ground vehicle’s (UGV’s) navigation task. The framework is applied for model validation of Global Positioning System (GPS), inertial measurement unit (IMU), and RGB camera sensor models. The framework is further applied to validate these sensor models for an example camera-based autonomous navigation algorithm, stop sign detection.