Efficacy, usability and acceptability of a chatbot for promoting COVID-19 vaccination in unvaccinated or booster-hesitant young adults: pre-post study (Preprint)

BACKGROUND COVID-19 vaccines are highly effective in preventing severe disease and death but are underused. Interventions to address COVID-19 vaccine hesitancy are paramount to reducing the burden of COVID-19. OBJECTIVE The primary aim is to evaluate the efficacy, usability, and acceptability of a chatbot for promoting COVID-19 vaccination. The secondary aim is to examine the factors associated with COVID-19 vaccine hesitancy. METHODS In November 2021, we conducted a pre-post pilot study to evaluate the “Vacc chat, Fact check”, a web-based chatbot for promoting COVID-19 vaccination. To recruit participants, we conducted an online survey (N=290) on COVID-19 vaccination at a university in Hong Kong. A subset of 46 participants who were either unvaccinated (N=22) or completed vaccination but hesitant to receive booster (N=24) were selected and given access to the chatbot for a 7-day trial period. The chatbot provided information about COVID-19 vaccination (e.g., efficacy, common side effects), debunked common myths about the vaccine and included a decision aid for selecting vaccine platforms (inactivated and mRNA vaccines). The main outcome was changes in COVID-19 vaccine hesitancy scale (VHS) score (range 9 to 45) from preintervention (online survey) to postintervention (immediately after the trial). Other outcomes included changes in intention to vaccinate/ receive booster and willingness to encourage others to vaccinate (range 1 to 5). Usability was assessed by the System Usability Scale (range 0 to 100). Linear regression was used to examine the factors associated with COVID-19 VHS scores in all survey respondents. RESULTS The mean (SD) age of all survey respondents was 21.4 (6.3) years; 61% (177/290) were female. Higher eHealth literacy (B = -0.26; P<.001) and perceived danger of COVID-19 (B = -0.17; P=.009) were associated with lower COVID vaccine hesitancy, adjusting for age, sex, chronic disease status, previous flu vaccination, and perceived susceptibility to COVID-19. The main outcome of COVID-19 VHS score significantly reduced from 28.6 (preintervention) to 24.5 (postintervention), with a mean difference of -4.2 (P<.001) and effect size (Cohen’s d) of 0.94. The intention to vaccinate increased from 3.0 to 3.9 (P<.001) in unvaccinated participants, while the intention to receive boosters increased from 1.9 to 2.8 (P<.001) in booster-hesitant participants. Willingness to encourage others to vaccinate increased from 2.7 to 3.0 (P=.038). At post-intervention, the median (IQR) System Usability Scale Score was 72.5 (65–77.5), while the median (IQR) recommendation score was 7 (6–8) on a scale of 0 to 10. In a post hoc 4-month follow-up, 82% (18/22) of initially unvaccinated participants reported having received the COVID-19 vaccine, while 29% (7/24) booster-hesitant participants received boosters. CONCLUSIONS This pilot study provided initial evidence to support the efficacy, usability and acceptability of a chatbot for promoting COVID-19 vaccination in young adults who were unvaccinated or booster-hesitant.