O-11 Using the think aloud method to evaluate instrument design for a new survey of patient engagement with advance care planning (ACP)

Background Various survey design problems may introduce error and hinder the measurement of patient engagement with Advance Care Planning (ACP). The think aloud method allows for clarification of instructions, constructs, items, survey logic and flow. Aim To apply the use of “think aloud methods” to detect design issues in a new survey developed to measure patient engagement with ACP. Methods Patients with chronic diseases related to renal function (n = 4), cardiac function (n = 3), and cancer (n = 7), as well as healthier patients at family practice clinics (n = 7) were recruited to “think out loud” when completing a new survey evaluating patient engagement with ACP. Survey modes included hardcopy or electronic using an iPad. Think aloud interviews were audio-recorded and analysed using the constant comparison method. Three reviewers independently listened to the interviews, summarised findings and discussed discrepancies until consensus was achieved. Results Three rounds of interviews were needed until saturation for patient clarity was achieved. We found issues related to constructs, response options, instructions and language pertaining to patient engagement in ACP. Most patients were aware of ACP in the legal domain as opposed to the healthcare domain. Those with chronic diseases were likely to have an ACP agent. Discussion The think aloud method was useful in refining the survey instrument. It also revealed patient confusion about the documents used in Alberta to record ACP decisions while raising patient awareness. The findings will help establish the validity of the new survey. Conclusion This method was useful for instrument design while also providing information about how patients engage with ACP.