SavedAdd your journal

Journals / Acta Medica Alanya / 2024 / Cilt: 8 Sayı: 3

Is ChatGPT a Useful Tool for Ophthalmology Practice?

ChatGPT Oftalmoloji Pratiğinde Faydalı Bir Araç Mıdır?

Fuat Yavrum(ALANYA ALAADDİN KEYKUBAT ÜNİVERSİTESİ)0000-0002-0708-5508
Dilara Özkoyuncu(TOBB UNIVERSITY OF ECONOMICS AND TECHNOLOGY)0000-0001-5196-0106
Pages
221–227
Publication Date
2024-12-30
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Results: ChatGPT achieved an overall accuracy of 56.1% in case-based questions.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Aim: This study aimed to assess ChatGPT-3.5's performance in ophthalmology, comparing its responses to clinical case-based and multiple-choice (MCQ) questions.
📊
Key Findings & Data:Methods: ChatGPT-3.5, an AI model developed by OpenAI, was employed.
💡
Conclusion & Impact:Results: ChatGPT achieved an overall accuracy of 56.1% in case-based questions.
🔬Design:Review & Systematic Survey
📍Scope:Comprehensive Sample
⚙️Variables:Yapay zeka · ChatGPT · Büyük dil modeli

Abstract

Aim: This study aimed to assess ChatGPT-3.5's performance in ophthalmology, comparing its responses to clinical case-based and multiple-choice (MCQ) questions. Methods: ChatGPT-3.5, an AI model developed by OpenAI, was employed. It responded to 98 case-based questions from "Ophthalmology Review: A Case-Study Approach" and 643 MCQs from "Review Questions in Ophthalmology" book. ChatGPT's answers were compared to the books, and statistical analysis was conducted. Results: ChatGPT achieved an overall accuracy of 56.1% in case-based questions. Accuracy varied across categories, with the highest in the retina section (69.5%) and the lowest in the trauma section (38.2%). In MCQ, ChatGPT's accuracy was 53.5%, with the weakest in the optics section (32.6%) and the highest in pathology and uveitis (66.7% and 63.0%, respectively). ChatGPT performed better in case-based questions in the retina and pediatric ophthalmology sections than MCQ. Conclusion: ChatGPT-3.5 exhibits potential as a tool in ophthalmology, particularly in retina and pediatric ophthalmology. Further research is needed to evaluate ChatGPT's clarity and acceptability for open-ended questions.

Özet

Amaç: ChatGPT-3.5'in performansını göz hastalıkları alanında değerlendirmek, klinik vaka bazlı sorular ve çoktan seçmeli sorulara (ÇSS) verdiği yanıtların doğruluk oranını karşılaştırmaktır. Yöntem: Çalışmada OpenAI tarafından geliştirilen bir yapay zeka modeli olan ChatGPT-3.5 kullanıldı. Modelden, "Ophthalmology Review: A Case-Study Approach" kitabından 98 vaka bazlı soruya ve "Review Questions in Ophthalmology" kitabından 643 ÇSS'ye yanıt vermesi istendi. ChatGPT'nin cevapları kitaplarla karşılaştırıldı ve istatistiksel analizi yapıldı. Bulgular: ChatGPT, vaka bazlı sorularda genel olarak %56,1 doğruluk oranı gösterdi.. Doğruluk oranı kategoriler arasında en yüksek retina bölümünde (%69,5) ve en düşük travma bölümünde (%38,2) idi. ÇSS'de ChatGPT'nin genel doğruluk oranı %53,5 olarak gözlendi, bunların en düşüğü optik bölümünde (%32,6) ve en yükseği patoloji ve üveit bölümlerinde (%66,7 ve %63) idi. ChatGPT özellikle retina ve pediatrik oftalmoloji bölümlerindeki vaka bazlı sorularda ÇSS’ye kıyasla daha iyi performans gösterdi. Sonuç: ChatGPT-3.5, özellikle retina ve pediatrik oftalmoloji alanlarında göz hastalıkları için potansiyel bir yardımcı araç olarak görülmektedir. ChatGPT'nin açık uçlu sorular için netlik ve kabul edilebilirliğini değerlendirmek için daha fazla araştırma yapılması gerekmektedir.

🔖

Cite This Article

Yavrum, F., Özkoyuncu, D. (2024). Is ChatGPT a Useful Tool for Ophthalmology Practice?. Acta Medica Alanya, Cilt: 8 Sayı: 3, 221–227. https://doi.org/10.30565/medalanya.1531790

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (17)

  1. Li JO, Liu H, Ting DSJ, Jeon S, Chan RVP, et al. Digital technology, tele-medicine and artificial intelligence in ophthalmology: A global perspective. Prog Retin Eye Res. 2021;82:100900. doi: 10.1016/j.preteyeres.2020.100900.
  2. Sallam M. ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare (Basel). 2023;11(6):887. doi: 10.3390/healthcare11060887.
  3. Introducing ChatGPT. https://openai.com/blog/chatgpt Accessed May 17, 2023.
  4. Alkaissi H, McFarlane SI. Artificial Hallucinations in ChatGPT: Implications in Scientific Writing. Cureus. 2023;15(2):e35179. doi: 10.7759/cureus.35179.
  5. Gilson A, Safranek CW, Huang T, Socrates V, Chi L, et al. How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med Educ. 2023;9:e45312. doi: 10.2196/57594.
  6. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. doi: 10.1371/journal.pdig.0000198.
  7. Cai LZ, Shaheen A, Jin A, Fukui R, Yi JS, et al. Performance of Generative Large Language Models on Ophthalmology Board-Style Questions. Am J Ophthalmol. 2023;254:141-9. doi: 10.1016/j.ajo.2023.05.024.
  8. Raumviboonsuk P, Krause J, Chotcomwongse P, Sayres R, Raman R, et al. Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program. NPJ Digit Med. 2019;2:25. doi: 10.1038/s41746-019-0099-8.
  9. Bogunovic H, Montuoro A, Baratsits M, Karantonis MG, Waldstein SM, et al. Machine Learning of the Progression of Intermediate Age-Related Macular Degeneration Based on OCT Imaging. Invest Ophthalmol Vis Sci. 2017;58(6):BIO141-50. doi: 10.1167/iovs.17-21789.
  10. Singh K, Smiddy WE, Lee AG. Ophthalmology review: a case-study approach. Second edition. Thieme; 2018
  11. Kenneth C. Chern, Michael A. Saidel. Review Questions in Ophthalmology. Third edition. Wolters Kluwer; 2014
  12. Antaki F, Touma S, Milad D, El-Khoury J, Duval R. Evaluating the Performance of ChatGPT in Ophthalmology: An Analysis of Its Successes and Shortcomings. Ophthalmol Sci. 2023;3(4):100324. doi: 10.1016/j.xops.2023.100324.
  13. Mihalache A, Popovic MM, Muni RH. Performance of an Artificial Intelligence Chatbot in Ophthalmic Knowledge Assessment. JAMA Ophthalmol. 2023;141(6):589-97. doi: 10.1001/jamaophthalmol.2023.1144.
  14. Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172-180. doi: 10.1038/s41586-023-06291-2.
  15. Moshirfar M, Altaf AW, Stoakes IM, Tuttle JJ, Hoopes PC. Artificial Intelligence in Ophthalmology: A Comparative Analysis of GPT-3.5, GPT-4, and Human Expertise in Answering StatPearls Questions. Cureus. 2023;15(6):e40822. doi: 10.7759/cureus.40822.
  16. Stunkel L, Mackay DD, Bruce BB, Newman NJ, Biousse V. Referral Patterns in Neuro-Ophthalmology. J Neuroophthalmol. 2020;40(4):485-93. doi: 10.1097/WNO.0000000000000846.
  17. Singh K, Smiddy WE, Lee AG. Ophthalmology review: a case-study approach. Second edition. Thieme; 2018