Urological Cancers and ChatGPT: Assessing the Quality of Information and Possible Risks for Patients

This study assesses ChatGPT's efficacy in addressing inquiries on urological cancers, focusing on prostate, kidney, bladder, and testicular cancers. Frequently asked questions from diverse sources and the EAU 2023 Guideline Oncology were used for evaluation. ChatGPT 4.0 premium version responses were appraised using the global quality score (GQS). Results indicate high GQS scores for general queries but lower scores for EAU guideline-related questions, emphasizing ChatGPT's commendable accuracy for broad inquiries and areas for improvement in guideline-specific responses. Introduction: OpenAI has created ChatGPT, an artificial intelligence language model that has gained considerable recognition for its capacity to produce text responses resembling human language. Consequently, this study seeks to evaluate the effectiveness of ChatGPT's responses in addressing publicly accessible queries related to prostate, kidney, bladder, and testicular cancers. Material and Methods: A comprehensive compilation of frequently asked questions (FAQs) pertaining to prostate, bladder, kidney, and testicular cancers was gathered from diverse sources. Additionally, the recommendations outlined in the European Association of Urology (EAU) 2023 Guideline Oncology were consulted. The chosen questions for evaluation were presented to the ChatGPT 4.0 premium version. The quality of ChatGPT responses was appraised using the global quality score (GQS). Each ChatGPT response was independently reviewed by a panel of physicians, who assigned a GQS score to assess its overall quality. Results: For prostate cancer, 64.6% of the questions had a GQS score of 5, compared to 62.9 % for bladder, 68.1% for kidney, and 63.9% for testicular cancers, whereas none of the responses had a GQS score of 1. Meanwhile, the category with the lowest proportion of responses, with a GQS score of 5 for each disease, was prognosis and follow-up. The mean GQS score of the answers given to EAU guideline questions was statistically significantly lower than the average score of the answers given to FAQs. Conclusion: ChatGPT is a valuable tool for addressing general inquiries regarding urological cancers, boasting commendable accuracy rates. Nonetheless, its performance in responding to questions aligned with the EAU guideline was deemed unsatisfactory.