Asking the right questions: Developing diagnostic tests in undergraduate physics

Being able to discover students' conceptions and more importantly alternate- and misconceptions about a topic is vital in order to be able to assess and thus be able to improve student learning. It is well known that this can be achieved via the use of well- designed diagnostic tests, a widely used example of which is the Force Concept Inventory. Creating the right questions in order to form a reliable diagnostic test can be a lengthy and complicated process. This article reports work on a Development Project funded in 2008 to develop such a test for introductory Quantum Mechanics courses in both physics and chemistry. We present details of our methodology, which involves augmenting a 'standard' multiple-choice question set with free-response boxes to determine the reasons for a student choosing a particular answer, and a self-assessment of their level of confidence in their choice. The responses from piloting this initial test in different institutions are used to inform the subsequent refinement of the test, as well as assessing the reliability and validity of the questions. We highlight examples of misconceptions that have been found during the development of the diagnostic tests.