An Exploratory Study of a Novel Adaptive e-Learning Board Review Product Helping Candidates Prepare for Certification Examinations

This article was migrated. The article was not marked as recommended. Objectives: Online board-review products are widely used to prepare for high-stakes examinations; however, it is unclear if these products are truly helpful and which features are most useful. This study examined the user ratings of an e-learning examination-preparation product and its impact on first-time American Board of Internal Medicine Certifying Examination (ABIM-CE) test takers. Methods: 4,400 users were anonymously surveyed between 2015 and 2017 to determine their level of satisfaction with the NEJM Knowledge+ Internal Medicine Board Review (NEJM Knowledge+) product and their board-exam passage rate. Results:The responding users' (n=836) average ratings of NEJM Knowledge+'s content, relevance, components, and features were high. Similarly, a subgroup of responding users from the United States who primarily used NEJM Knowledge+ to prepare for an initial ABIM-CE between 2014 and 2016 (n=153) rated the content, relevance, components, and features high, with practice exams, adaptive delivery of questions, and reports on progress, performance, and metacognition as the most valuable features. Importantly, this subgroup of users passed the ABIM-CE on their first attempt at a significantly higher rate than the national average (95% vs 89%, z=2.6397, p=0.0083). Conclusion:An e-learning examination-preparation product that features adaptive delivery of questions, offers detailed feedback, and analyzes overall metacognition may be an effective tool for first-time test takers preparing for initial certifying examinations.