Smart Enough to Pass Exam Questions

In two recent studies published in Radiology, researchers evaluated ChatGPT’s performance in answering radiology board exam questions. While the AI showed potential, it also demonstrated limitations affecting its reliability. ChatGPT, based on GPT-3.5, answered 69% of questions correctly, struggling more with higher-order thinking questions due to its lack of radiology-specific training.

A subsequent study with GPT-4 showcased improvement, answering 81% correctly and excelling in higher-order thinking questions. However, it still faced reliability concerns, answering some questions incorrectly and exhibiting occasional inaccuracies termed “hallucinations.”

Confident language was consistently used, even in incorrect responses, posing a risk, especially for novices who might not recognize inaccuracies.

Decision Making in Cancer Screening: Bard Vs ChatGPT

A study recently published in American Radiology compares ChatGPT-4 and Bard, two large language models, in aiding radiology decisions for breast, ovarian, colorectal, and lung cancer screenings. They tested various prompts, finding both models to perform well overall. ChatGPT-4 showed higher accuracy in certain scenarios, especially with ovarian cancer screening. However, Bard performed better with specific prompts for breast and colorectal cancer. Open-ended prompts improved both models’ performance, suggesting their potential use in unique clinical scenarios. The study acknowledged limitations in scoring subjectivity, limited scorers, and the focus on specific cancer screenings based on ACR guidelines.

Simplifying Readability of Reports

The study in European Radiology explores using ChatGPT and similar large language models to simplify radiology reports for easier patient comprehension. Researchers had ChatGPT translate complex reports into simpler language for patient understanding. Fifteen radiologists evaluated these simplified reports, finding them generally accurate and complete, yet also identified factual errors and potentially misleading information in a significant portion of the simplified reports. Despite these issues, the study highlights the potential for large language models to enhance patient-centered care in radiology and other medical fields, emphasizing the need for further adaptation and oversight to ensure accuracy and patient safety.

Sources:

Rsna.org
diagnosticimaging.com
Radiologybusiness.com
openai.com

The post A Look at 2023 and ChatGPT In Radiology first appeared on Vesta Teleradiology.

How Could ChatGPT be Used?

An article in Diagnostic and Interventional Imaging discusses various ways in which radiologists can leverage ChatGPT. It highlights applications for clinical radiologists, such as implementing ChatGPT as a chatbot for patient inquiries, supporting clinical decision-making with information and analysis assistance, and enhancing patient communication and follow-up care by simplifying radiology reports and crafting tailored recommendations. Academic radiologists can benefit from ChatGPT by receiving suggestions for impactful research article titles, assistance with structuring and formatting academic papers, and help in formatting citations for bibliographies. The article emphasizes that the best use of ChatGPT in radiology depends on individual needs and goals, potentially paving the way for a more intelligent future in the field.  It notes that while ChatGPT offers valuable support, it’s crucial to fact-check its answers and review its output to ensure accuracy and relevance.

What Radiologists Have to Say

In RSNA’s article, The Good, the Bad and the Ugly of Using ChatGPT, various radiologists give their opinions on the use of this AI. Dr. Som Biswas, who published an article in Radiology entirely written by ChatGPT, believes that its potential benefits in reducing the workload and improving efficiency in radiology outweigh its limitations, which could be especially valuable in addressing the growing demand for medical imaging and reports in the face of a radiologist shortage.

Yiqiu Shen, MS, a researcher at New York University’s Center for Data Science, remarked, “In general, it’s ok to use ChatGPT as a language aid or to provide a template, but it’s dangerous to rely on ChatGPT to make a clinical decision.”

Urologic Imaging and AI: A Study

A study published in Current Problems in Diagnostic Radiology compared the performance of OpenAI’s ChatGPT and Google Bard in suggesting appropriate urologic imaging methods based on American College of Radiology (ACR) criteria. Both chatbots demonstrated an appropriate imaging modality rate of over 60%, with no significant difference between them in the proportion of correct imaging modality selected. However, the researchers noted that both chatbots lacked consistent accuracy and further development is needed for clinical implementation. The study found that while the chatbots were not entirely consistent in their responses, they hold promise in assisting healthcare providers in determining the best imaging modality, potentially improving clinical workflows in the future. ChatGPT provided shorter responses and had a slightly longer response time compared to Bard, which was faster but struggled with determining appropriate imaging modalities in a few scenarios.

Vesta: A Tech-Forward Company

Vesta Teleradiology looks forward to a future integrating AI with medicine. Click here to read more about Vesta Teleradiology Partners with MIT for AI Research

Sources:

radiologybusiness.com
rsna.org
Auntminnie.com
openai.com

The post ChatGPT in Radiology: Is it a Pro or Con? first appeared on Vesta Teleradiology.