Journals / Ağrı Tıp Fakültesi Dergisi / 2026 / Cilt: 4 Sayı: 3

Comparison of ChatGPT models' responses on lateral epicondylitis in terms of reliability and usefulness

ChatGPT modellerinin lateral epikondilit üzerine yanıtlarının güvenilirlik ve yararlılık açısından karşılaştırılması

Emre Uzun(Sakarya Eğitim ve Araştırma Hastanesi)0000-0002-4394-1906
Muhammed Zahid Şahin(SAKARYA UNIVERSITY, SCHOOL OF MEDICINE, MEDICINE PR.)0009-0004-8639-2742
Rıdvan Işık(Sakarya Training and Research Hospital)0000-0002-5715-2844
İbrahim Tekeoğlu(KUTAHYA HEALTH SCIENCES UNIVERSITY)0000-0003-3395-7178
Ayhan Kamanli(SAKARYA UNIVERSITY, SCHOOL OF MEDICINE, MEDICINE PR.)0000-0001-5299-7250
Pages
99–105
Publication Date
2026-10-02
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

These findings support ChatGPT’s potential as patient-education aid for musculoskeletal disorders while underscoring the need for expert oversight and further research across broader topics, languages, and chatbot platforms.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Aim: Our aim in this study was to compare the reliability and clinical usefulness of answers generated by ChatGPT’s GPT-5.1 Instant and GPT-5.1 Thinking models for the most frequently searched patient questions on lateral epicondylitis.
📊
Key Findings & Data:The six highest-ranked questions were posed to GPT-5.1 Instant and GPT-5.1 Thinking.
💡
Conclusion & Impact:These findings support ChatGPT’s potential as patient-education aid for musculoskeletal disorders while underscoring the need for expert oversight and further research across broader topics, languages, and chatbot platforms.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Tenisçi dirseği · Yapay zeka · Konu olarak hasta eğitimi

Abstract

Aim: Our aim in this study was to compare the reliability and clinical usefulness of answers generated by ChatGPT’s GPT-5.1 Instant and GPT-5.1 Thinking models for the most frequently searched patient questions on lateral epicondylitis. Materials and Methods: This study was designed as a pilot, descriptive, expert-evaluated large language model (LLM) output assessment study. Google Trends was queried for “tennis elbow” and “lateral epicondylitis”. The six highest-ranked questions were posed to GPT-5.1 Instant and GPT-5.1 Thinking. Three senior physiatrists independently rated every response using the seven-point Reliability and Usefulness Scales. Results: Both models produced reliable and moderately useful responses. Mean ± SD reliability scores were 5.11 ± 1.18 for GPT-5.1 Instant and 5.06 ± 1.21 for GPT-5.1 Thinking, while usefulness scores were 5.22 ± 1.22 and 5.50 ± 1.38, respectively. Differences were not statistically significant (reliability Z = -1.00, p = 0.317, r ≈ 0.41; usefulness Z = -1.60, p = 0.109, r ≈ 0.65), though GPT-5.1 Thinking trended higher in usefulness. Inter-rater agreement was excellent for all ratings (ICC = 0.94-0.98), indicating consistent scoring. Conclusion: GPT-5.1 Instant and GPT-5.1 Thinking both delivered reliable, moderately useful information on lateral epicondylitis, with GPT-5.1 Thinking showing a numerically higher usefulness score that did not reach statistical significance. These findings support ChatGPT’s potential as patient-education aid for musculoskeletal disorders while underscoring the need for expert oversight and further research across broader topics, languages, and chatbot platforms.

Özet

Amaç: Bu çalışmanın amacı, lateral epikondilit ile ilgili hastaların en sık aradığı sorulara ChatGPT’nin GPT-5.1 Instant ve GPT-5.1 Thinking modelleri tarafından verilen yanıtların güvenilirlik ve klinik yararlılık açısından karşılaştırılmasıdır. Gereçler ve Yöntemler: Bu çalışma, pilot, tanımlayıcı, uzman değerlendirmeli bir büyük dil modeli (LLM) çıktı değerlendirme çalışması olarak tasarlandı. Google Trends’de “tenisçi dirseği” ve “lateral epikondilit” terimleri sorgulandı. En yüksek sıralamaya sahip altı soru belirlendi ve her biri Türkçe olarak GPT-5.1 Instant ve GPT-5.1 Thinking modellerine yöneltildi. Üç kıdemli fiziksel tıp ve rehabilitasyon uzmanı tüm yanıtları yedi puanlı Güvenilirlik ve Yararlılık Ölçeklerini kullanarak bağımsız biçimde değerlendirdi. Bulgular: Her iki model de güvenilir ve orta düzeyde yararlı yanıtlar üretti. Ortalama ± SS güvenilirlik skorları GPT-5.1 Instant için 5.11 ± 1.18, GPT-5.1 Thinking için 5.06 ± 1.21 olarak bulundu. Yararlılık skorları sırasıyla 5.22 ± 1.22 ve 5.50 ± 1.38 idi. Farklar istatistiksel olarak anlamlı değildi (güvenilirlik: Z = -1.00, p = 0.317, r ≈ 0.41; yararlılık: Z = -1.60, p = 0.109, r ≈ 0.65), ancak GPT-5.1 Thinking yararlılık açısından daha yüksek eğilim gösterdi. Değerlendiriciler arası uyum tüm ölçümlerde mükemmeldi (ICC = 0.94-0.98). Sonuç: GPT-5.1 Instant ve GPT-5.1 Thinking, lateral epikondilit hakkında güvenilir ve orta derecede yararlı bilgiler sunmuştur. GPT-5.1 Thinking modelinde yararlılık skoru sayısal olarak daha yüksek bulunmuş, ancak bu fark istatistiksel olarak anlamlı değildir. Bu bulgular, ChatGPT’nin kas-iskelet sistemi hastalarında hasta eğitimi için potansiyel bir araç olabileceğini, ancak uzman denetimi ve daha geniş konu, dil ve chatbot platformlarını içeren ileri araştırmaların gerekli olduğunu göstermektedir.

🔖

Cite This Article

Uzun, E., Şahin, M. Z., Işık, R., Tekeoğlu, İ., Kamanli, A. (2026). Comparison of ChatGPT models' responses on lateral epicondylitis in terms of reliability and usefulness. Ağrı Tıp Fakültesi Dergisi, Cilt: 4 Sayı: 3, 99–105. https://doi.org/10.61845/agrimedical.1859819

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (17)

  1. Bhabra G, Wang A, Ebert JR, Edwards P, Zheng M, Zheng MH. Lateral elbow tendinopathy: Development of a pathophysiology-based treatment algorithm. Orthop J Sports Med. 2016;4(11):2325967116670635. https://doi.org/10.1177/2325967116670635
  2. Cutts S, Gangoo S, Modi N, Pasapula C. Tennis elbow: A clinical review article. J Orthop. 2020;17:203-7. https://doi.org/10.1016/j.jor.2019.08.005.
  3. Chu JT, Wang MP, Shen C, Viswanath K, Lam TH, Chan SSC. How, when and why people seek health information online: Qualitative study in Hong Kong. Interact J Med Res. 2017;6(2):e24. https://doi.org/10.2196/ijmr.7000.
  4. Shahsavar Y, Choudhury A. User intentions to use ChatGPT for self-diagnosis and health-related purposes: Cross-sectional survey study. JMIR Hum Factors. 2023;10:e47564. https://doi.org/10.2196/47564.
  5. Alelyani T. Establishing trust in artificial intelligence-driven autonomous healthcare systems: An expert-guided framework. Front Digit Health. 2024;6:1474692. https://doi.org/10.3389/fdgth.2024.1474692
  6. Chen S, Kann BH, Foote MB, Aerts HJWL, Savova GK, Mak RH, et al. Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncol. 2023;9(10):1459-62. https://doi.org/10.1001/jamaoncol.2023.2954
  7. Uz C, Umay E. “Dr ChatGPT”: Is it a reliable and useful source for common rheumatic diseases? Int J Rheum Dis. 2023;26(7):1343-9. https://doi.org/10.1111/1756-185X.14749.
  8. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-63. https://doi.org/10.1016/j.jcm.2016.02.012.
  9. Wei Q, Yao Z, Cui Y, Wei B, Jin Z, Xu X. Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis. J Biomed Inform. 2024;151:104620. https://doi.org/10.1016/j.jbi.2024.104620.
  10. Rossettini G, Bargeri S, Cook C, Guida S, Palese A, Rodeghiero L, et al. Accuracy of ChatGPT-3.5, ChatGPT-4o, Copilot, Gemini, Claude, and Perplexity in advising on lumbosacral radicular pain against clinical practice guidelines: Cross-sectional study. Front Digit Health. 2025;7:1574287. https://doi.org/10.3389/fdgth.2025.1574287.
  11. Yan Z, Liu J, Fan Y, Lu S, Xu D, Yang Y, et al. Ability of ChatGPT to replace doctors in patient education: Cross-sectional comparative analysis of inflammatory bowel disease. J Med Internet Res. 2025;27:e62857. https://doi.org/10.2196/62857
  12. Shrestha N, Shen Z, Zaidat B, Duey AH, Tang JE, Ahmed W, et al. Performance of ChatGPT on NASS clinical guidelines for the diagnosis and treatment of low back pain: A comparison study. Spine (Phila Pa 1976). 2024;49(9):640-51. https://doi.org/10.1097/BRS.0000000000004915
  13. Ahmed W, Saturno M, Rajjoub R, Duey AH, Zaidat B, Hoang T, et al. ChatGPT versus NASS clinical guidelines for degenerative spondylolisthesis: A comparative analysis. Eur Spine J. 2024;33(11):4182-203. https://doi.org/10.1007/s00586-024-08198-6
  14. Gianola S, Bargeri S, Castellini G, Cook C, Palese A, Pillastrini P, et al. Performance of ChatGPT compared to clinical practice guidelines in making informed decisions for lumbosacral radicular pain: A cross-sectional study. J Orthop Sports Phys Ther. 2024;54(3):222-8. https://doi.org/10.2519/jospt.2024.12151
  15. Karataş Ö, Demirci S, Pota K, Tuna S. Assessing ChatGPT’s role in sarcopenia and nutrition: Insights from a descriptive study on AI-driven solutions. J Clin Med. 2025;14(5):1747. https://doi.org/10.3390/jcm14051747
  16. Keles A, Illeez OG, Erbagci B, Giray E. Artificial intelligence-generated responses to frequently asked questions on coccydynia: Evaluating the accuracy and consistency of GPT-4o’s performance. Arch Rheumatol. 2025;40(1):63-71. https://doi.org/10.46497/ArchRheumatol.2025.10966
  17. Rocha-Silva R, Rodrigues MAM, Viana RB, Nakamoto FP, Vancini RL, Andrade MS, et al. Critical analysis of information provided by ChatGPT on lactate, exercise, fatigue, and muscle pain: Current insights and future prospects for enhancement. Adv Physiol Educ. 2024;48(4):898-903. https://doi.org/10.1152/advan.00073.2024