ChatGPT has quickly moved beyond its niche beginnings and become an integral part of everyday life. Its reach extends well past casual conversation, now penetrating various industries, notably the intricate world of radiology. As we close out 2023, we take a look at some headlines that show how far ChatGPT has advanced in the realm of diagnostic imaging.
Smart Enough to Pass Exam Questions
In two recent studies published in Radiology, researchers evaluated ChatGPT’s performance in answering radiology board exam questions. While the AI showed potential, it also demonstrated limitations affecting its reliability. ChatGPT, based on GPT-3.5, answered 69% of questions correctly, struggling more with higher-order thinking questions due to its lack of radiology-specific training.
A subsequent study with GPT-4 showcased improvement, answering 81% correctly and excelling in higher-order thinking questions. However, it still faced reliability concerns, answering some questions incorrectly and exhibiting occasional inaccuracies termed “hallucinations.”
Confident language was consistently used, even in incorrect responses, posing a risk, especially for novices who might not recognize inaccuracies.
Decision Making in Cancer Screening: Bard Vs ChatGPT
A study recently published in American Radiology compares ChatGPT-4 and Bard, two large language models, in aiding radiology decisions for breast, ovarian, colorectal, and lung cancer screenings. They tested various prompts, finding both models to perform well overall. ChatGPT-4 showed higher accuracy in certain scenarios, especially with ovarian cancer screening. However, Bard performed better with specific prompts for breast and colorectal cancer. Open-ended prompts improved both models’ performance, suggesting their potential use in unique clinical scenarios. The study acknowledged limitations in scoring subjectivity, limited scorers, and the focus on specific cancer screenings based on ACR guidelines.
Simplifying Readability of Reports
The study in European Radiology explores using ChatGPT and similar large language models to simplify radiology reports for easier patient comprehension. Researchers had ChatGPT translate complex reports into simpler language for patient understanding. Fifteen radiologists evaluated these simplified reports, finding them generally accurate and complete, yet also identified factual errors and potentially misleading information in a significant portion of the simplified reports. Despite these issues, the study highlights the potential for large language models to enhance patient-centered care in radiology and other medical fields, emphasizing the need for further adaptation and oversight to ensure accuracy and patient safety.
Sources:
Rsna.org
diagnosticimaging.com
Radiologybusiness.com
openai.com