Pediatric tele-coaching fidelity evaluation: Feasibility, perceived satisfaction and usefulness of a new measure

BackgroundTo promote and ensure coaches' fidelity in delivering an online health coaching program to parents of children with suspected developmental delay, we developed and implemented a novel coaching fidelity rating tool, CO-FIDEL (COaches Fidelity in Intervention DELivery). We aimed to (1) Demonstrate CO-FIDEL's feasibility in evaluating coaches' fidelity and its change over time; and (2) Explore coaches' satisfaction with and usefulness of the tool.MethodsIn an observational study design, coaches (n = 4) were assessed using the CO-FIDEL following each coaching session (n = 13-14 sessions/parent-participant) during the pilot phase of a large randomized clinical trial involving eleven (n = 11) parent-participants. Outcome measures included subsections' fidelity measures, overall coaching fidelity, and coaching fidelity changes over time analyzed using descriptive and non-parametric statistics. In addition, using a four-point Likert Scale and open-ended questions, coaches were surveyed on their satisfaction and preference levels, as well as facilitators, barriers, and impacts related to the use of CO-FIDEL. These were analyzed using descriptive statistics and content analysis.ResultsOne hundred and thirty-nine (n = 139) coaching sessions were evaluated with the CO-FIDEL. On average, overall fidelity was high (88.0 & PLUSMN; 6.3 to 99.5 & PLUSMN; 0.8%). Four coaching sessions were needed to achieve and maintain a & GE; 85.0% fidelity in all four sections of the tool. Two coaches showed significant improvements in their coaching skills over time in some of the CO-FIDEL sections (Coach B/Section 1/between parent-participant B1 and B3: 89.9 & PLUSMN; 4.6 vs. 98.5 & PLUSMN; 2.6, Z = -2.74, p = 0.00596; Coach C/Section 4/between parent-participant C1 and C2: 82.4 & PLUSMN; 7.5 vs. 89.1 & PLUSMN; 4.1, Z = -2.66; p = 0.00758), and in overall fidelity (Coach C, between parent-participant C1 and C2: 88.67 & PLUSMN; 6.32 vs. 94.53 & PLUSMN; 1.23, Z = -2.66; p = 0. 00758). Coaches mainly reported moderate-high satisfaction with and usefulness of the tool, and pointed out areas of improvement (e.g., ceiling effect, missing elements).ConclusionsA new tool ascertaining coaches' fidelity was developed, applied, and shown to be feasible. Future research should address the identified challenges and examine the psychometric properties of the CO-FIDEL.