Validation testing to determine viability and application of a translation device in radiotherapy: Presenter(s): Darren Hunter, Monash University, Australia

Quality communication is critical to the provision of optimal care in radiotherapy. Bi-directional language allows for informed consent, information provision, disclosure of treatment-related health concerns, and active treatment decision making. Globalisation has driven multicultural growth, placing greater demand on multilingual support services. Health systems commonly utilise professional interpreters to overcome language barriers, though issues of access are prevalent. Concurrent development of machine translation technology presents opportunity for an accessible and cost-effective supplement. This study aimed to develop and implement validation methodology to determine clinical suitability and output reliability for radiotherapy application. The study focused solely on English-Mandarin translation – the most common languages spoken in Australian society. This study comprised four domains: 1) critical appraisal of available translator devices, 2) compilation of common phrases, 3) participant recruitment, and 4) device performance testing. Pre-determined English phrases were spoken into the device, with the output Mandarin scored independently by two interpreters utilising a 5-point Likert scale. Analysis entailed descriptive and inferential statistics – employing Pearson’s chi-square test, single proportion hypothesis test and linear regression modelling. A market scan yielded four potential translation devices for review – Travis Translator progressed to validation testing. 188 phrases were collated by the project team; categorized by conversation type and readability score. 1128 validation trials were conducted with six participants. An overall pass rate of 66% (n = 744) was observed – significantly greater than prior research. Uniformity of interpreter scores were evident in 63% of trials; a further 29% constituted a single point variation. Device accuracy improved with repeated use, though colloquialisms and lower readability scores attracted poorer outcomes. This study presents a novel validation methodology for the determination of accuracy and reliability of machine translation. Future research should consider clinical feasibility, training requirements, and expansion to other languages or health disciplines.