Open-ended comments in SIC-Ex, an assessment tool for residents leading Serious Illness Conversations

Abstract Purpose The Serious Illness Conversation (SIC) has emerged as a framework for conversations with patients with a serious illness diagnosis. This study reports on narratives generated from open-ended questions of a novel assessment tool, the SIC-Evaluation Exercise (SIC-Ex), to assess resident-led conversations with patients in oncology outpatient clinics. Methods We developed the SIC-Ex based on the Ariadne SIC framework. Seven resident trainees and ten preceptors were recruited from three cancer centres. Each trainee conducted a SIC with a patient, which was videotaped. The preceptors watched the videos and evaluated each trainee using the novel SIC-Ex and the reference Calgary-Cambridge Guide (CCG) at months 0 and 3. Two independent coders used template analysis to code the preceptors’ free-text narrative comments and identify themes/subthemes. Results Template analysis yielded 6 themes: behavioural attributes mapped to SIC, those mapped to CCG, those overlapping between SIC and CCG, trainees’ demeanors, rater mis-classification of comments, and comments on SIC-Ex. Narrative comments explored numerous verbal and non-verbal components essential to SIC. Some comments applied to both SIC and CCG (e.g. setting agenda, introduction, planning, exploring, non-verbal communication), whereas others mapped specific to one (e.g. SIC - identifying substitute decision maker, affirming commitment, introducing advance care planning, engaging family; CCG – using open ended questions, avoiding explanations, flow, time management). Conclusion Narrative comments generated by SIC-Ex provided a detailed and nuanced insight into trainee's competency in SIC, beyond numerical ratings and general communication skills assessed by CCG; they should continue to be a part of assessment.