Skill Mastery Measurement and Prediction to Adapt Instruction Strategies.

In this paper we present the design, development, application and validation of our skill mastery measurement software to help instructors adapt their instruction strategy to teach Visual AirCraft recognition (VACR) skill. Visual AirCraft recognition (VACR) is a critical skill required by soldiers operating surface-to-air missile defense systems to quickly recognize the aircraft and make engagement decisions. The current computer-based trainer to teach VACR has no intelligence to help instructors deliver personalized instructions or to track skill gaps. We compare our candidate automated support system that utilizes Dynamic Bayesian Network (DBN) models to measure skill progression to other conventional tutors and report our findings in terms of improved skill transfer and retention. The instruction delivery format incorporated in our automated support system is equivalent to Wings, Engine, Fuselage and Tail (WEFT) training that is provided by the instructors in a classroom setting. The goal of this study is to measure utility of our skill-tracking based automated system prior to its adoption by VACR instructors to impart WEFT training at Air Defense Artillery, Fort Sill, OK. We performed an Amazon Mechanical Turk (AMT) study to compare efficacy of our DBN-based VACR trainer with baseline and incremental tutors. The results indicate that our system increased overall transfer and retention performance of AMT participants by 19% and 16%, respectively compared to other tutors. We present the rationale behind design of these candidate tutors, implementation of DBN model for VACR and the AMT study design with final results.