Development of an assessment tool to measure communication skills among family medicine residents in the context of electronic medical record use

Background The introduction of the electronic medical record (EMR) has led to new communication skills that need to be taught and assessed. There is scarce literature on validated instruments measuring electronic-specific communication skills. The aim is to develop an assessment checklist that assesses the general and EMR-specific communication skills and evaluates their content validity and reliability. Methods Using the SEGUE theoretical framework for communication skills, the assessment checklist items were developed by the Communication Skills Working Group (CSWG) at the family medicine department using a literature review about the positive and negative aspects of EMR use on physician-patient communication. A group of faculty members rated real resident-patient encounters on two occasions, three weeks apart. Patients were asked to fill out the Communication Assessment Tool (CAT) at the end of the encounter. Results A total of 8 residents agreed to participate in the research, with 21 clinical encounters recorded. The average total score was 65.2 ± 6.9 and 48.1 ± 9.5 for the developed scale and the CAT scale, respectively. The scale reliability was good, with a Cronbach alpha of 0.694. The test-retest reliability was 0.873, p < 0.0001. For the total score on the developed checklist, the intraclass correlation coefficient between raters (ICC) was 0.429 [0.030,0.665], p-value of 0.019. The level of agreement between any two raters on the cumulative score of the 5 subsections ranged from 0.506 (interpersonal skills) to 0.969 (end encounter). Conclusion This checklist is a reliable and valid instrument that combines basic and EMR-related communication skills.