Artificial Intelligence Chatbots for medical training in Public Health: a tool to explore

Abstract Background Artificial intelligence (AI) is a rapidly developing field with the potential to transform various aspects of healthcare and public health including medical training. During the ‘Hygiene and Public Health’ course for fifth-year medical students, a practical training session was conducted on vaccination using AI as an educational supportive tool. Before receiving specific training on vaccination, the students were given an online questionnaire on vaccination. After completing the questionnaire, a correction of each question was performed using AI chatbots. Objectives The main aim was to identify whether AI chatbots can be considered as educational support tool for training in public health. Secondary objective was to assess performance of 6 different AI chatbot models. Results The questionnaire consisted of 15 items selected from the Italian National Medical Residency Test (years 2015-2022) on the topic of vaccination. 36 medical students participated in the training, achieving a mean score of 8.22/15 on the quiz (range:3-15,median:8). 6 different AI chatbot models (3 modes of BingChat, ChatGPT, ChatSonic, and YouChat) were asked the same questions and achieved a mean score of 12.17/15 (range:10-15,median:11.5), ranking in the top 25% of medical student scores. During the class, the answers provided by the AI chatbots were evaluated with the students, paying particular attention to the completeness of the information, reliability of the sources cited, and use of technical language. Conclusions AI chatbots have shown to be efficient tools for retrieving bibliographic references from the web and answering questions related to specific topics such as vaccination. However, it is important to note that they cannot be considered more intelligent than students, as they may not perform as well when faced with situational questions. The responsible and critical use of AI should be integrated into medical education to ensure its effective utilization. Key messages • AI chatbots use as support tool in public health medical training should be further investigated and endorsed. • AI chatbots could be a helpful resource for medical students in terms of researching and revising their knowledge.