SavedAdd your journal

Journals / Uluslararası Diş Hekimliği Bilimleri Dergisi / 2026 / Cilt: 12 Sayı: 2

Comparative Evaluation of Large Language Models as Virtual Orthodontic Patient Information Assistants

Büyük Dil Modellerinin Sanal Ortodonti Hasta Bilgilendirme Asistanları Olarak Karşılaştırmalı Değerlendirilmesi

Alperen Erdoğan(ZONGULDAK BÜLENT ECEVİT ÜNİVERSİTESİ)0009-0007-5024-706X
Orhan Çiçek(ZONGULDAK BULENT ECEVİT UNIVERSITY)0000-0002-8172-6043
Pages
82–89
Publication Date
2026-08-31
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Conclusion: This study demonstrates that LLMs can serve as auxiliary tools in providing orthodontic patient information.However, responses should be checked by an expert before being presented to patients and should be adapted into simpler and more understandable language.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Objective: The aim of this study was to evaluate the potential use of widely used large language models (LLMs) in patient education by comparing their responses to orthodontic patient questions in terms of accuracy, comprehensiveness, and readability.
📊
Key Findings & Data:Methods: This study evaluated the performance of Claude Sonnet 4.6, ChatGPT 5.5 and Gemini 3.5 Pro in providing orthodontic patient information.Ten frequently asked questions by orthodontic patients were posed to each model in independent sessions using the same wording.
💡
Conclusion & Impact:Conclusion: This study demonstrates that LLMs can serve as auxiliary tools in providing orthodontic patient information.However, responses should be checked by an expert before being presented to patients and should be adapted into simpler and more understandable language.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:yapay zeka · büyük dil modelleri · doğruluk

Abstract

Objective: The aim of this study was to evaluate the potential use of widely used large language models (LLMs) in patient education by comparing their responses to orthodontic patient questions in terms of accuracy, comprehensiveness, and readability. Methods: This study evaluated the performance of Claude Sonnet 4.6, ChatGPT 5.5 and Gemini 3.5 Pro in providing orthodontic patient information.Ten frequently asked questions by orthodontic patients were posed to each model in independent sessions using the same wording. The responses obtained were scored by two orthodontists for accuracy and comprehensiveness using a modified Likert scale. The readability of the responses was calculated using the FRE and FKGL indices. Differences between the models were evaluated with statistical analyses appropriate for the data structure (p < 0.05). Results: No significant difference was found between the models regarding readability. However, significant differences were found in accuracy and comprehensiveness scores. Gemini 3.5 Pro achieved the highest scores in terms of comprehensiveness and accuracy showed significant superiority over ChatGPT 5.5 (p = 0.007). Conclusion: This study demonstrates that LLMs can serve as auxiliary tools in providing orthodontic patient information.However, responses should be checked by an expert before being presented to patients and should be adapted into simpler and more understandable language.

Özet

Amaç: Bu çalışmanın amacı, yaygın kullanılan büyük dil modellerinin ortodontik hasta sorularına verdikleri yanıtları doğruluk, kapsamlılık ve okunabilirlik açısından karşılaştırarak hasta bilgilendirmede kullanım potansiyellerini değerlendirmektir. Yöntem: Bu araştırmada ChatGPT 5.5, Claude Sonnet 4.6 ve Gemini 3.5 Pro’nun ortodontik hasta bilgilendirmesindeki performansları değerlendirildi. Ortodontik hastaların sık sorduğu 10 soru, her modele bağımsız oturumlarda ve aynı ifadelerle yöneltildi. Elde edilen yanıtlar iki ortodontist tarafından doğruluk ve kapsamlılık açısından modifiye Likert ölçeğiyle puanlandı. Yanıtların okunabilirliği ise FRE ve FKGL indeksleri kullanılarak hesaplandı. Modeller arasındaki farklar, veri yapısına uygun istatistiksel analizlerle değerlendirildi (p < 0.05). Bulgular: Modeller arasında okunabilirlik açısından anlamlı fark saptanmadı. Buna karşın doğruluk ve kapsamlılık puanlarında anlamlı farklılık bulundu. Gemini 3.5 Pro, doğruluk ve kapsamlılık açısından en yüksek skorları elde etti ve ChatGPT 5.5’e göre anlamlı üstünlük gösterdi (p = 0.007). Sonuç: Bu çalışma, büyük dil modellerinin ortodontik hasta bilgilendirmesinde yardımcı bir araç olarak kullanılabileceğini göstermektedir. Ancak yanıtlar hastalara sunulmadan önce uzman tarafından kontrol edilmeli ve daha sade, anlaşılır bir dile uyarlanmalıdır.

🔖

Cite This Article

Erdoğan, A., Çiçek, O. (2026). Comparative Evaluation of Large Language Models as Virtual Orthodontic Patient Information Assistants. Uluslararası Diş Hekimliği Bilimleri Dergisi, Cilt: 12 Sayı: 2, 82–89. https://doi.org/10.21306/dishekimligi.1994307

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (25)

  1. Draelos RL, Afreen S, Blasko B, Brazile TL, Chase N, Desai DP, et al. Large language models provide unsafe answers to patient-posed medical questions. NPJ digital medicine. 2026;9(1):241.
  2. Asgari E, Montaña-Brown N, Dubois M, Khalil S, Balloch J, Yeung JA, Pimenta D. A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation. NPJ digital medicine. 2025;8(1):274.
  3. Kong M, Fok EHW, Yiu CKY. A Scoping Review of Large Language Models in Dental Education: Applications, Challenges, and Prospects. Int Dent J. 2025;75(6):103854.
  4. Farhadi Nia M, Ahmadi M, Irankhah E. Transforming dental diagnostics with artificial intelligence: advanced integration of ChatGPT and large language models for patient care. Frontiers in Dental Medicine. 2025;5:1456208.
  5. Akitomo T, Hamada M, Hamaguchi S, Kusaka S, Nomura R. Generative Artificial Intelligence and Large Language Models in Paediatric Dentistry: A Scoping Review. Int Dent J. 2026;76(2):109401.
  6. Gandhi S, Balas M. Algorithmic bias in clinical resource allocation by a large language model: a cross-sectional in-silico evaluation of 13,608 decisions. BMC Med Ethics. 2026;27(1):136.
  7. Cârstea A-E, Ciocan L-T, Vasilescu V-G, Țâncu A-MC, Imre M, Didilescu A-C, Pițuru S-M. Large Language Models in Medical and Dental Education: A Cross-Sectional Comparison of AIGenerated and Faculty-Authored Prosthodontic Materials. Dent J. 2026;14(5):249.
  8. Özkan T, Acar A, Özkan E, Düzyol M, Öztürk E. Are artificial intelligence based chatbots reliable sources for patients regarding orthodontics. APOS Trends Orthod. 2025;15(2):141-6.
  9. Cicek O, Yıldırım A. Evaluating the reliability of artificial intelligence in clinical decisions on early maxillary expansion: a comparative study of eight large language models. BMC Oral Health. 2026;26:715.
  10. Demir Cicek B, Cicek O. Evaluating the Response of AI-Based Large Language Models to Common Patient Concerns About Endodontic Root Canal Treatment: A Comparative Performance Analysis. J Clin Med 2025;14(21):7482.
  11. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105-11.
  12. Dobbs Ferry N. A new readability yardstick. J Appl Psychol. 1948;32(3):221-33.
  13. Kincaid JP, Fishburne Jr RP, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. 1975.
  14. Piersson AD, Ofori-Manteaw B, Mumuni HN, Dzefi-Tettey K. Readability of patient education materials on cardiac magnetic resonance imaging. Eur Heart J Imaging Methods Pract. 2025;3(2):qyaf111.
  15. Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172- 80.
  16. Wei Q, Yao Z, Cui Y, Wei B, Jin Z, Xu X. Evaluation of ChatGPT-generated medical responses: a systematic review and meta-analysis. J Biomed Inform. 2024;151:104620.
  17. Dursun D, Bilici Geçer R. Can artificial intelligence models serve as patient information consultants in orthodontics? BMC Med Inform Decis Mak. 2024;24(1):211.
  18. Hatia A, Doldo T, Parrini S, Chisci E, Cipriani L, Montagna L, et al. Accuracy and completeness of ChatGPT-generated information on interceptive orthodontics: a multicenter collaborative study. J Clin Med. 2024;13(3):735.
  19. Vassis S, Powell H, Petersen E, Barkmann A, Noeldeke B, Kristensen KD, et al. Cureus. 2024;16(8):e68085.
  20. Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med. 2023;183(6):589-96.
  21. Albalawi F, Khanagar SB, Iyer K, Alhazmi N, Alayyash A, Alhazmi AS, et al. Evaluating the performance of artificial intelligence-based large language models in orthodontics—a systematic review and meta-analysis. Appl Sci. 2025;15(2):893.
  22. Sato H, Ogasawara K, Sakurai H. Performance Evaluation of 18 Generative AI Models (ChatGPT, Gemini, Claude, and Perplexity) in 2024 Japanese Pharmacist Licensing Examination: Comparative Study. JMIR Med Educ. 2025;11(1):e76925.
  23. Roustan D, Bastardot F. The clinicians’ guide to large language models: a general perspective with a focus on hallucinations. Interact J Med Res. 2025;14(1):e59823.
  24. Gracea RS, Winderickx N, Vanheers M, Hendrickx J, Preda F, Shujaat S, et al. Artificial intelligence for orthodontic diagnosis and treatment planning: A scoping review. J Dent. 2025;152:105442.
  25. Özden S, Erener H. Comparative Evaluation of ChatGPT-4o, Gemini 2.5 Pro and Grok-4 in Answering Orthodontics Questions from the Dentistry Specialty Examination. Lokman Hekim Health Sci. 2026;6(1):126-33.