Predicting Team Performance Through Human Behavioral Sensing and Quantitative Workflow Instrumentation

For decades, the social sciences have provided the foundation for the study of humans interacting with systems; however, sparse, qualitative, and often subjective observations can be insufficient in capturing the complex dynamics of modern sociotechnical enterprises. Technical advances in quantitative system-level and physiological instrumentation have made possible greater objective study of human-system interactions, and joint qualitative-quantitative methodologies are being developed to improve human performance characterization. In this paper we detail how these methodologies were applied to assess teams’ abilities to effectively discover information, collaborate, and make risk-informed decisions during serious games. Statistical models of intra-game performance were developed to determine whether behaviors in specific facets of the gameplay workflow were predictive of analytical performance and game outcomes. A study of over seventy instrumented teams revealed that teams who were more effective at face-to-face communication and system interaction performed better at information discovery tasks and had more accurate game decisions.