SavedAdd your journal

Journals / Osmangazi Tıp Dergisi / 2026 / Cilt: 48 Sayı: 6

Patient Education in Hand Surgery: A Blinded Comparison of GPT-5.5 and Claude Opus 4.8

El Cerrahisinde Hasta Eğitimi: GPT-5.5 ile Claude Opus 4.8'in Kör Karşılaştırması

İbrahim Faruk Adıgüzel(Etlik City hospital)0000-0003-2493-5540
Hakan Ertem(İNÖNÜ ÜNİVERSİTESİ, TIP FAKÜLTESİ)0000-0002-1724-4138
Pages
1072–1078
Publication Date
2026-09-22
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

We compared the reliability, overall quality, and readability of patient-facing texts generated by GPT-5.5 and Claude Opus 4.8.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Patients increasingly consult conversational artificial intelligence about hand and wrist conditions, but the performance of proprietary large language models remains unclear.
📊
Key Findings & Data:We compared the reliability, overall quality, and readability of patient-facing texts generated by GPT-5.5 and Claude Opus 4.8.
💡
Conclusion & Impact:Because FKGL and FRE quantify surface linguistic complexity rather than understanding, these findings describe written outputs only; they do not establish better patient comprehension, clinical effectiveness, factual equivalence, or safety.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Yapay Zekâ · Büyük Dil Modelleri · Hasta Eğitimi

Abstract

Patients increasingly consult conversational artificial intelligence about hand and wrist conditions, but the performance of proprietary large language models remains unclear. We compared the reliability, overall quality, and readability of patient-facing texts generated by GPT-5.5 and Claude Opus 4.8. Thirty hand surgery topics were selected from orthopedic education resources and the authors' outpatient practice. Each model received the same prompt and generated one response per topic (60 texts). Two fellowship-trained, certified hand surgeons independently evaluated source-masked outputs. DISCERN assessed reliability and treatment information, the Global Quality Score (GQS) overall quality, and the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) readability. Platforms were compared with the Mann-Whitney U test and interrater agreement with the intraclass correlation coefficient (ICC). Median DISCERN scores were 66.5 for GPT-5.5 and 63.5 for Claude Opus 4.8 (p = 0.143); median GQS was 4.0 for both (p = 0.389). Claude Opus 4.8 had a lower median FKGL (8.75 vs 10.4; p < 0.001) and higher median FRE (50.35 vs 43.6; p = 0.009). ICCs (95% CIs) were 0.755 (0.589-0.853) for DISCERN and 0.676 (0.458-0.807) for GQS. Expert-rated reliability and overall quality did not differ significantly, whereas Claude Opus 4.8 produced more favorable formula-based readability scores; median FKGL for both models nevertheless remained above the recommended sixth-grade level. Because FKGL and FRE quantify surface linguistic complexity rather than understanding, these findings describe written outputs only; they do not establish better patient comprehension, clinical effectiveness, factual equivalence, or safety.

Özet

Hastalar el ve el bileği hastalıkları hakkında bilgi edinmek için giderek daha fazla konuşmaya dayalı yapay zekâ araçlarına başvurmaktadır. Bununla birlikte, mevcut büyük dil modellerinin performansı yeterince tanımlanmamıştır. Bu çalışmada GPT-5.5 ve Claude Opus 4.8 tarafından üretilen hasta eğitimi metinlerinin güvenilirliği, genel kalitesi ve okunabilirliği karşılaştırıldı. Ortopedik eğitim kaynakları ve yazarların poliklinik pratiğine dayanarak 30 el cerrahisi konusu seçildi. Her model aynı istemle konu başına bir yanıt üretti (toplam 60 metin). Yan dal eğitimi almış iki el cerrahı, kaynak kimliği maskelenmiş metinleri bağımsız değerlendirdi. Güvenilirlik ve tedavi bilgisi DISCERN, genel kalite Global Kalite Skoru (GQS), okunabilirlik Flesch-Kincaid Sınıf Düzeyi (FKGL) ve Flesch Okuma Kolaylığı (FRE) ile değerlendirildi. Platformlar Mann-Whitney U testiyle, değerlendiriciler arası uyum sınıf içi korelasyon katsayısı (ICC) ile analiz edildi. Ortanca DISCERN puanı GPT-5.5 için 66,5, Claude Opus 4.8 için 63,5 idi (p = 0,143); ortanca GQS her iki modelde 4,0 bulundu (p = 0,389). Claude Opus 4.8 daha düşük FKGL (8,75'e karşı 10,4; p < 0,001) ve daha yüksek FRE (50,35'e karşı 43,6; p = 0,009) sağladı. ICC nokta tahminleri (%95 GA) DISCERN için 0,755 (0,589-0,853), GQS için 0,676 (0,458-0,807) idi. Uzmanlarca puanlanan güvenilirlik ve genel kalite bakımından anlamlı fark saptanmazken Claude Opus 4.8 daha olumlu formüle dayalı okunabilirlik göstergeleri ortaya koydu; her iki modelde de ortanca FKGL, önerilen altıncı sınıf okuma düzeyinin üzerinde bulundu. FKGL ve FRE yüzeysel dilsel karmaşıklığı ölçtüğünden bu bulgular daha iyi hasta anlayışının, klinik etkinliğin, olgusal eşdeğerliğin veya güvenliğin kanıtı olarak yorumlanmamalıdır.

🔖

Cite This Article

Adıgüzel, İ. F., Ertem, H. (2026). Patient Education in Hand Surgery: A Blinded Comparison of GPT-5.5 and Claude Opus 4.8. Osmangazi Tıp Dergisi, Cilt: 48 Sayı: 6, 1072–1078. https://doi.org/10.20515/otd.2004895

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (28)

  1. Ozkan S, Mellema JJ, Nazzal A, Lee SG, Ring D. Online health information seeking in hand and upper extremity surgery. J Hand Surg Am 2016;41(12):e469-e475.
  2. Daraz L, Morrow AS, Ponce OJ, et al. Readability of online health information: a meta-narrative systematic review. Am J Med Qual 2018;33(5):487-492.
  3. Aydin S, Karabacak M, Vlachos V, Margetis K. Large language models in patient education: a scoping review of applications in medicine. Front Med (Lausanne) 2024;11:1477898.
  4. Jagiella-Lodise O, Suh N, Zelenski NA. Can patients rely on ChatGPT to answer hand pathology-related medical questions? Hand (N Y) 2025;20(5):801-809.
  5. Croen BJ, Abdullah MS, Berns E, et al. Evaluation of patient education materials from large-language artificial intelligence models on carpal tunnel release. Hand (N Y) 2025;20(6):893-899.
  6. Parmar RP, Daulat SR, Shah R, Brady TT, Montague M, Roth C. Readability, accuracy, and lexical diversity of new ChatGPT models for common carpal tunnel syndrome questions. Hand (N Y). Published online February 8, 2026.
  7. Pohl NB, Derector E, Rivlin M, et al. A quality and readability comparison of artificial intelligence and popular health website education materials for common hand surgery procedures. Hand Surg Rehabil 2024;43(3):101723.
  8. Gorgos P, Ternell KH, Hammarstrand C, et al. ChatGPT and Claude in hand surgery: an explanatory evaluation of clinical decision support on common surgical cases. Hand Surg Rehabil 2025;44(6):102530.
  9. Bayrak HC, Karagöz B, Bayrak Ö. Comparative evaluation of large language model-based chatbots in a septic arthritis scenario: ChatGPT, Claude, and Perplexity. Acta Orthop Traumatol Turc 2025;59(6):415-420.
  10. American Academy of Orthopaedic Surgeons. OrthoInfo: hand and wrist conditions. https://orthoinfo.aaos.org/diseases--conditions/?bodyPart=HandWrist. Accessed June 1, 2026.
  11. Orthobullets. Hand specialty dashboard. https://www.orthobullets.com/topic/dashboard?id=6&specialty=6. Accessed June 1, 2026.
  12. OpenAI. Introducing GPT-5.5. https://openai.com/index/introducing-gpt-5-5/. Accessed July 15, 2026.
  13. Anthropic. Introducing Claude Opus 4.8. https://www.anthropic.com/news/claude-opus-4-8. Accessed July 15, 2026.
  14. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53(2):105-111.
  15. Bernard A, Langille M, Hughes S, Rose C, Leddin D, Veldhuyzen van Zanten S. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol 2007;102(9):2070-2077.
  16. Kincaid JP, Fishburne RP Jr, Rogers RL, Chissom BS. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report 8-75. Memphis, TN: Chief of Naval Technical Training, Naval Air Station Memphis; 1975.
  17. Flesch R. A new readability yardstick. J Appl Psychol 1948;32(3):221-233.
  18. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15(2):155-163.
  19. Gezer MC, Armangil M. Assessing the quality of ChatGPT's responses to commonly asked questions about trigger finger treatment. Ulus Travma Acil Cerrahi Derg 2025;31(4):389-393.
  20. Casey JC, Wang L, Milner JD, et al. ChatGPT is a concise, comprehensive, and accurate alternative to Google for answering common hand therapy questions. Hand Ther. Published online April 21, 2026.
  21. Eltorai AEM, Sharma P, Wang J, Daniels AH. Most American Academy of Orthopaedic Surgeons' online patient education material exceeds average patient reading level. Clin Orthop Relat Res 2015;473(4):1181-1186.
  22. Ghanem D, Covarrubias O, Maxson R, Sabharwal S, Shafiq B. Readability of trauma-related patient education materials from the American Academy of Orthopaedic Surgeons and Orthopaedic Trauma Association websites. J Am Acad Orthop Surg 2024;32(13):e642-e650.
  23. Yuncu M, Kose O. Comparative evaluation of large language models for patient education after total knee arthroplasty. Knee 2026;61:104445.
  24. Travis LM, Prasad S, Deiparine S, Marmor WA, Rizzo MG. Evaluation of ChatGPT-4o as a patient information tool for common orthopaedic surgeries: accuracy, completeness, and clinical utility. JAAOS Glob Res Rev 2025;9(12):e25.00341.
  25. Fiedler B, Ghilzai U, Ghali A, et al. A supplement, not a substitute: accuracy and completeness of ChatGPT responses for common elbow pathology. Shoulder Elbow 2026;18(2):379-385.
  26. Olszewski R, Brzezinski J, Watros K, Rysz J. Quantifying readability in chatbot-generated medical texts using classical linguistic indices: a review. Appl Sci 2026;16(3):1423.
  27. Will J, Gupta M, Zaretsky J, Dowlath A, Testa P, Feldman J. Enhancing the readability of online patient education materials using large language models: cross-sectional study. J Med Internet Res 2025;27:e69955.
  28. Pohl NB, Tarawneh OH, Johnson E, Aita D, Tadley M, Fletcher DJ. Patient preferences for carpal tunnel release education: a comparison of education materials from popular healthcare websites and ChatGPT. Hand Surg Rehabil 2025;44(1):102073.