Automatic Generation of Just-in-time Online Assessments from Software Design Models

Computer software is pervasive in today's society. The rate at which new versions of computer software products are released is phenomenal when compared to the release rate of new products in traditional industries such as aircraft building. This rapid rate of change can partially explain why most certifications in the software industry are generic as opposed to those in the aircraft-building industry where engineers and technicians are certified to work on a specific aircraft. For example, a software engineer may be certified on a database management system, but not on a specific implementation based on the database management system. Hence, software engineers are allowed to make critical changes to specific designs for the next release of a software product with little formal assessment of their understanding of the design. This paper presents a system that automatically generates just-in-time online assessments for judging a software engineer's comprehension of artifacts representing software designs. The assessments thus generated are compliant with the IMS-QTI 2.1 standard. The system is based on the AXIS web-services architecture and provides a priori statistical estimates of effectiveness of each individually generated assessment.