Design feasibility of an automated, machine-learning based feedback system for motivational interviewing.

Direct observation of psychotherapy and providing performance-based feedback is the gold-standard approach for training psychotherapists. At present, this requires experts and training human coding teams, which is slow, expensive, and labor intensive. Machine learning and speech signal processing technologies provide a way to scale up feedback in psychotherapy. We evaluated an initial proof of concept automated feedback system that generates motivational interviewing quality metrics and provides easy access to other session data (e.g.. transcripts). The system automatically provides a report of session-level metrics (e.g., therapist empathy) and therapist behavior codes at the talk-turn level (e.g., reflections). We assessed usability, therapist satisfaction, perceived accuracy, and intentions to adopt. A sample of 21 novice (n = 10) or experienced (n = 11) therapists each completed a 10-min session with a standardized patient. The system received the audio from the session as input and then automatically generated feedback that therapists accessed via a web portal. All participants found the system easy to use and were satisfied with their feedback, 83% found the feedback consistent with their own perceptions of their clinical performance, and 90% reported they were likely to use the feedback in their practice. We discuss the implications of applying new technologies to evaluation of psychotherapy.