SavedAdd your journal

Journals / Endoüroloji Bülteni / 2026 / Cilt: 18 Sayı: 1

A Comparison Study of AI Response Quality in Kidney Stone-Related Questions: Google vs. ChatGPT

Böbrek Taşıyla İlgili Sorularda Yapay Zekâ (ChatGPT) ve Google Cevap Kalitesinin Karşılaştırılması

Burak Elmaağaç(SAĞLIK BİLİMLERİ ÜNİVERSİTESİ, KAYSERİ ŞEHİR SAĞLIK UYGULAMA VE ARAŞTIRMA MERKEZİ, CERRAHİ TIP BİLİMLERİ BÖLÜMÜ, ÜROLOJİ ANABİLİM DALI)0000-0001-5853-0256
Abdullah Gölbaşı(SAĞLIK BİLİMLERİ ÜNİVERSİTESİ, KAYSERİ SAĞLIK UYGULAMA VE ARAŞTIRMA MERKEZİ)0000-0001-8397-975X
Ali Yasin Özercan(Kayseri Şehir Hastanesi)0000-0001-8378-0409
Hüseyin Biçer(Kayseri Şehir Hastanesi)0000-0001-7703-3618
Pages
30–38
Publication Date
2026-01-29
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

These results show the possibilities of artificial intelligence language models in access to correct healthcare information and patient education.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Objective: This study intends to evaluate, in Türkiye during the previous 12 months, the readability, reliability, and general quality of responses given by Google and ChatGPT on often-requested inquiries about kidney stones.
📊
Key Findings & Data:Material And Methods: The 10 most often searched queries and 4 trending topics on kidney stones were found by means of Google Trends data.
💡
Conclusion & Impact:These results show the possibilities of artificial intelligence language models in access to correct healthcare information and patient education.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:böbrek taşları · hasta eğitimi · internet

Abstract

Objective: This study intends to evaluate, in Türkiye during the previous 12 months, the readability, reliability, and general quality of responses given by Google and ChatGPT on often-requested inquiries about kidney stones. Material And Methods: The 10 most often searched queries and 4 trending topics on kidney stones were found by means of Google Trends data. Fourteen queries in all were asked on Google and ChatGPT. Gunning Fog Index (readability), a modified 5-question DISCERN Score (reliability), the Global Quality Score, and expert urologists’ assessments guided responses’ analysis. Independent t-tests let one evaluate Google’s and ChatGPT’s performance differences. Results: ChatGPT showed notably better performance than Google in terms of the DISCERN Score (mean: 21.4 vs. 18.0; p = 0.002), Global Quality Score (mean: 4.8 vs. 4.0; p = 0.001), and expert urologists’ ratings (mean: 4.6 vs. 3.8; p = 0.001). Though ChatGPT had a lower Gunning Fog Index (mean: 15.84 vs. 18.23), indicating more legible text, this difference was not statistically significant (p = 0.165). Conclusion: ChatGPT exceeded Google in offering consistent, high-quality, expert-endorsed knowledge about kidney stones. These results show the possibilities of artificial intelligence language models in access to correct healthcare information and patient education. Additionally, improving transparency of information sources and implementing further verification mechanisms for clinical accuracy are recommended to ensure the reliability of AI-generated medical information.

Özet

Amaç: Türkiye’de böbrek taşıyla ilgili sık aranan sorularda Google ve ChatGPT’nin verdiği yanıtların okunabilirlik, güvenilirlik ve genel kalite açısından karşılaştırılması. Gereç ve Yöntemler: Google Trends ile 2023 yılı için en sık aranan 10 soru ve ani yükseliş gösteren 4 konu belirlendi (toplam 14 sorgu). Google için ilk sayfa özetleri, ChatGPT (GPT-4) için doğrudan yanıtlar değerlendirildi. Okunabilirlik Gunning Fog İndeksi; güvenilirlik 5 maddelik modifiye DISCERN; genel kalite Global Quality Score ile puanlandı. Üç deneyimli üroloji uzmanı ayrıca klinik fayda ve doğruluk açısından 1–5 arası puan verdi. Bulgular: ChatGPT, DISCERN (21,4 vs. 18,0; p = 0,002), Global Quality Score (4,8 vs. 4,0; p = 0,001) ve uzman değerlendirmelerinde (4,6 vs. 3,8; p = 0,001) Google’dan üstün bulundu. Okunabilirlik ChatGPT’de daha iyi olsa da (15,84 vs. 18,23) fark istatistiksel olarak anlamlı değildi (p = 0,165). Sonuç: ChatGPT, böbrek taşıyla ilgili çevrim içi sağlık bilgisinin güvenilirliği ve genel kalitesi açısından Google’a göre daha tutarlı ve yüksek kaliteli içerik sunmaktadır. AI tabanlı dil modelleri hasta eğitimi ve sağlık okuryazarlığının desteklenmesinde potansiyele sahiptir; ancak kaynak şeffaflığının artırılması ve klinik doğruluk için ek doğrulama mekanizmaları önerilir.

🔖

Cite This Article

Elmaağaç, B., Gölbaşı, A., Özercan, A. Y., Biçer, H. (2026). A Comparison Study of AI Response Quality in Kidney Stone-Related Questions: Google vs. ChatGPT. Endoüroloji Bülteni, Cilt: 18 Sayı: 1, 30–38. https://doi.org/10.54233/endourolbull-1769794

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (20)

  1. Eysenbach G. The impact of the Internet on health professionals and patients: pros and cons. BMJ. 2001;323(7327):731-3. https://doi.org/10.1136/bmj.323.7327.731
  2. Morahan-Martin JM. How Internet users find, evaluate, and use online health information: a cross-cultural review. Cyberpsychol Behav. 2004;7(5):497-510. https://doi.org/10.1089/cpb.2004.7.497
  3. Brown T, Mann B, Ryder N, et al. Language models are few-shot learners. Adv Neural Inf Process Syst. 2020;33:1877-901.
  4. Hanna DR, Creswell ML, Terry RS, Vergamini LB, Sardiu M, et al. Bing chat for kidney stone management questions based on the AUA guidelines: a comparison of chatbot conversation style modes. World J Urol. 2025;43(1):151. https://doi.org/10.1007/s00345-025-05533-4
  5. Gençer Bingöl F, Ağagündüz D, Bingol MC. Accuracy of current large language models and the retrieval-augmented generation model in determining dietary principles in chronic kidney disease. J Ren Nutr. 2025;S1051-2276(25)00013-5. https://doi.org/10.1053/j.jrn.2025.01.004
  6. Aiumtrakul N, Thongprayoon C, Suppadungsuk S, Krisanapan P, Pinthusopon P, et al. Global trends in kidney stone awareness: a time series analysis from 2004-2023. Clin Pract (Basel). 2024;14(3):915-27. https://doi.org/10.3390/clinpract14030072
  7. Gunning R. The technique of clear writing. New York: McGraw-Hill; 1952.
  8. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105-11. https://doi.org/10.1136/jech.53.2.105
  9. Bernard A, Langille M, Hughes S, Rose C, Leddin D, Veldhuyzen van Zanten S. A systematic review of patient inflammatory bowel disease information on the Internet. Am J Gastroenterol. 2007;102(9):2070–77. https://doi.org/10.1111/j.1572-0241.2007.01325.x
  10. Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324(7337):573-7. https://doi.org/10.1136/bmj.324.7337.573
  11. Eysenbach G. Consumer health informatics. BMJ. 2000;320(7251):1713-6. https://doi.org/10.1136/bmj.320.7251.1713
  12. Temsah MH, Aljamaan F, Malki KH, Alhasan K, Altamimi I, et al. ChatGPT and the future of digital health: a study on healthcare workers’ perceptions and expectations. Healthcare (Basel). 2023;11(13):1812. https://doi.org/10.3390/healthcare11131812
  13. Nori H, King N, McKinney SM, Carignan D, Horvitz E. Capabilities of GPT-4 on medical challenge problems. arXiv Preprint. 2023. https://doi.org/10.48550/arXiv.2303.13375
  14. Sarikaya M, Ozcan Siki F, Ciftci I. Use of artificial intelligence in vesicoureteral reflux disease: a comparative study of guideline compliance. J Clin Med. 2025;14(7):2378. https://doi.org/10.3390/jcm14072378
  15. Jo E, Song S, Kim JH, Lim S, Kim JH, et al. Assessing GPT-4’s performance in delivering medical advice: comparative analysis with human experts. JMIR Med Educ. 2024;10:e51282. https://doi.org/10.2196/51282
  16. Vrdoljak J, Boban Z, Vilović M, Kumrić M, Božić J. A review of large language models in medical education, clinical decision support, and healthcare administration. Healthcare (Basel). 2025;13(6):603. https://doi.org/10.3390/healthcare13060603
  17. Çolakoğlu Y, Ayten A, Sertkaya C, Toksal K, Karadağ S. Evaluation of Chat generative pretrained transformer (ChatGPT) performance in answering kidney transplant related questions. New J Urol. 2025;20(1):21-3. https://doi.org/10.33719/nju1613084
  18. Lee J, Park J, Han HS. Using ChatGPT for kidney transplantation: perceived information quality by race and education levels. Clin Transplant. 2024;38(7):e15378. https://doi.org/10.1111/ctr.15378
  19. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44-56. https://doi.org/10.1038/s41591-018-0300-7
  20. Bahçeci T, Elmaağaç B, Ceyhan E. Comparative analysis of the effectiveness of Microsoft Copilot artificial intelligence chatbot and Google Search in answering patient inquiries about infertility: evaluating readability, understandability, and actionability. IJIR: Your Sexual Medicine Journal. 2025. https://doi.org/10.1038/s41443-025-01056-z