SavedAdd your journal

Journals / Journal of Medicine and Palliative Care / 2026 / Volume: 7 Issue: 3

Readability, reliability, and quality of AI chatbot responses on home mechanical ventilation

Ferhan Demirer Aydemir(CANAKKALE ONSEKIZ MART UNIVERSITY, SCHOOL OF MEDICINE)0000-0002-6740-1496
Volkan Hancı(DOKUZ EYLUL UNIVERSITY)0000-0002-2227-194X
Pages
—
Publication Date
2026-06-25
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Reliability was evaluated using the JAMA benchmark criteria and the modified DISCERN instrument, while overall educational quality was assessed using the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) tool.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Aims: Home mechanical ventilation (HMV) is an essential component of long-term respiratory care.
📊
Key Findings & Data:However, the readability, reliability, and quality of AI-generated information on HMV remain unclear.
💡
Conclusion & Impact:Reliability was evaluated using the JAMA benchmark criteria and the modified DISCERN instrument, while overall educational quality was assessed using the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) tool.
🔬Design:Modeling & Simulation
📍Scope:2025 Verileri
⚙️Variables:Artificial Intelligence · health literacy · home care services

Abstract

Aims: Home mechanical ventilation (HMV) is an essential component of long-term respiratory care. Patients and caregivers increasingly use online resources, including Artificial Intelligence (AI)-based chatbots, to obtain information regarding HMV. However, the readability, reliability, and quality of AI-generated information on HMV remain unclear. This study aimed to evaluate the readability, reliability, and quality of responses generated by ChatGPT, Gemini, and Perplexity for frequently asked questions about HMV. Methods: A cross-sectional content analysis was conducted in December 2025 using 47 high-interest HMV-related questions identified via Google Trends. Each question was entered verbatim into ChatGPT, Gemini, and Perplexity under standardized conditions. Readability was assessed using seven validated indices. Reliability was evaluated using the JAMA benchmark criteria and the modified DISCERN instrument, while overall educational quality was assessed using the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) tool. Non-parametric statistical tests were applied for inter-model comparisons. Results: All AI-generated responses significantly exceeded the recommended sixth-grade readability threshold across all indices (p<0.001). ChatGPT produced comparatively lower grade-level readability scores, whereas Perplexity demonstrated significantly higher reliability and quality scores. Gemini showed intermediate performance across most metrics. Significant inter-model differences were observed for multiple readability, reliability, and quality measures. A consistent association was identified between higher informational quality scores and increased textual complexity. Conclusion: AI-generated information on HMV is generally more complex than recommended for patients. Differences between platforms suggest that AI-generated medical content is not interchangeable and should be used only as a supplement to professional medical guidance.

🔖

Cite This Article

Aydemir, F. D., Hancı, V. (2026). Readability, reliability, and quality of AI chatbot responses on home mechanical ventilation. Journal of Medicine and Palliative Care, Volume: 7 Issue: 3. https://doi.org/10.47582/jompac.1949864

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (42)

  1. Simonds AK. Home mechanical ventilation: an overview. Ann Am Thorac Soc. 2016;13(11):2035-2044. doi:10.1513/AnnalsATS.201606-454FR
  2. MacIntyre EJ, Asadi L, McKim DA, Bagshaw SM. Clinical outcomes associated with home mechanical ventilation: a systematic review. Can Respir J. 2016;2016:6547180. doi:10.1155/2016/6547180
  3. Płaszewska-Żywko L, Fajfer-Gryz I, Cichoń J, Kózka M. Burden, social support, and coping strategies in family caregivers of individuals receiving home mechanical ventilation: a cross-sectional study. BMC Nurs. 2024;23(1):346. doi:10.1186/s12912-024-02024-6
  4. McKim DA, Road J, Avendano M, et al. Home mechanical ventilation: a Canadian Thoracic Society clinical practice guideline. Can Respir J. 2011;18(4):197-215. doi:10.1155/2011/139769
  5. Lipprandt M, Liedtke W, Langanke M, Klausen A, Baumgarten N, Röhrig R. Causes of adverse events in home mechanical ventilation: a nursing perspective. BMC Nurs. 2022;21(1):264. doi:10.1186/s12912-022-01038-2
  6. Clark M, Bailey S. Chatbots in health care: connecting patients to information: emerging health technologies. Ottawa (ON): Canadian Agency for Drugs and Technologies in Health; January 2024.
  7. Iqbal U, Tanweer A, Rahmanti AR, Greenfield D, Lee LTJ, Li YCJ. Impact of large language model (ChatGPT) in healthcare: an umbrella review and evidence synthesis. J Biomed Sci. 2025;32(1):45. doi:10.1186/s12929-025-01131-z
  8. Huo B, Boyle A, Marfo N, et al. Large language models for chatbot health advice studies: a systematic review. JAMA Netw Open. 2025;8(2): e2457879. doi:10.1001/jamanetworkopen.2024.57879
  9. Fahy S, Oehme S, Milinkovic D, Jung T, Bartek B. Assessment of quality and readability of information provided by ChatGPT in relation to anterior cruciate ligament injury. J Pers Med. 2024;14(1):104. doi:10.3390/jpm14010104
  10. Hanci V, Ergün B, Gül Ş, et al. Assessment of readability, reliability, and quality of ChatGPT®, Bard®, Gemini®, Copilot®, and Perplexity® responses on palliative care. Medicine (Baltimore). 2024;103(33):e39305. doi:10.1097/MD.0000000000039305
  11. Ozduran E, Hanci V. Evaluation of the 100 most cited articles on Sjogren's syndrome: a bibliometric analysis. J Basic Clin Health Sci. 2023;7:11-20. doi:10.30621/jbachs.1096610
  12. Ozduran E. YouTube as an information source for myofascial pain syndrome. In: Unlu Z, editor. Current approaches to myofascial pain syndrome. 1st ed. Ankara: Turkiye Klinikleri; 2026:p.377-382. doi:10.5336/978-625-395-854-1_p377
  13. Ozduran E, Hanci V. Exploring the reliability and accessibility of YouTube for cerebral palsy rehabilitation information. J Child. 2024;24: 15-24. doi:10.26650/jchild.2024.1301406
  14. Ozduran E, Hanci V. YouTube as a source of information about pulmonary rehabilitation during the COVID-19 pandemic. Eurasian J Pulmonol. 2022;24:107-114. doi:10.14744/ejp.2022.9221
  15. Ozbek IC, Ozduran E, Hanci V. Digital guidance: quality and readability analysis of artificial intelligence-generated spondyloarthropathy texts. Turk J Osteoporos. 2025;31:12-18. doi:10.4274/tod.galenos.2024.76743
  16. Mengi A, Ozduran E. Evaluating the readability, quality, and reliability of responses generated by artificial intelligence programs about acupuncture for pain. J Physical Med Rehab Sci. 2026;29:245-253. doi:10.31609/jpmrs.2025-115285
  17. Ozduran E, Hanci V. Digital rehabilitation in Parkinson's disease: the role of Artificial Intelligence-assisted exercise training. Turk J Osteoporos. 2025. doi:10.4274/tod.galenos.2025.66664
  18. Ozduran E, Hanci V. Evaluating the readability, quality, and reliability of online information on Sjogren's syndrome. Indian J Rheumatol. 2023; 18:16-25. doi:10.4103/injr.injr_56_22
  19. Li J, Dada A, Puladi B, Kleesiek J, Egger J. ChatGPT in healthcare: a taxonomy and systematic review. Comput Methods Programs Biomed. 2024;245:108013. doi:10.1016/j.cmpb.2024.108013
  20. Wei Q, Yao Z, Cui Y, Wei B, Jin Z, Xu X. Evaluation of ChatGPT-generated medical responses: a systematic review and meta-analysis. J Biomed Inform. 2024;151:104620. doi:10.1016/j.jbi.2024.104620
  21. Daram NR, Maxwell RA, D’Amato J, Massengill JC. Can Artificial Intelligence improve the readability of patient education information in gynecology? Am J Obstet Gynecol. 2025;233(6):640.e1-640.e9. doi:10.1016/j.ajog.2025.06.047
  22. Shukla IY, Sun MZ. Online and ChatGPT-generated patient education materials regarding brain tumor prognosis fail to meet readability standards. J Clin Neurosci. 2025;138:111410. doi:10.1016/j.jocn.2025.111410
  23. Stephenson-Moe CA, Behers BJ, Gibons RM, et al. Assessing the quality and readability of patient education materials on chemotherapy cardiotoxicity from Artificial Intelligence chatbots: an observational cross-sectional study. Medicine (Baltimore). 2025;104(15):e42135. doi:10.1097/MD.0000000000042135
  24. Nelson HC, Beauchamp MT, Pace AA. The reliability gap: how traditional search engines outperform Artificial Intelligence chatbots in rosacea public health information quality. Cureus. 2025;17(6):e86543. doi:10.7759/cureus.86543
  25. Altom DS, Taha AIA, Hussein AAAM, et al. Artificial Intelligence-based chatbots in chronic disease management: a systematic review of applications and challenges. Cureus. 2025;17(3):e81001. doi:10.7759/cureus.81001
  26. Khankeh HR, Ebadi A, Norouzi Tabrizi K, Moradian ST. Home health care for mechanical ventilation-dependent patients: a grounded theory study. Health Soc Care Community. 2022;30(5):e2157-e2168. doi:10.1111/hsc.13654
  27. Wasilewski MB, Kokorelias KM, Nonoyama M, et al. The experience of family caregivers of ventilator-assisted individuals who participated in a pilot web-based peer support program: a qualitative study. Digit Health. 2022;8:20552076221134964. doi:10.1177/20552076221134964
  28. Kara M, Ozduran E, Kara MM, Ozbek IC, Hanci V. Evaluating the readability, quality, and reliability of responses generated by ChatGPT, Gemini, and Perplexity on the most commonly asked questions about ankylosing spondylitis. PLoS One. 2025;20(6):e0326351. doi:10.1371/journal.pone.0326351
  29. Ozduran E. Evaluation of the readability of Turkish internet-based patient education materials related to low back pain. J Dokuz Eylul Univ Med Fac. 2022;36:135-150. doi:10.18614/deutip.1174522
  30. Ayers JW, Poliak A, Dredze M, et al. Comparing physician and Artificial Intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med. 2023;183(6):589-596. doi:10.1001/jamainternmed.2023.1838
  31. Traylor DO, Kern KV, Anderson EE, Henderson R. Beyond the screen: the impact of generative Artificial Intelligence on patient learning and the patient-physician relationship. Cureus. 2025;17(1):e76825. doi:10.7759/cureus.76825
  32. Daraz L, Morrow AS, Ponce OJ, et al. Can patients trust online health information? A meta-narrative systematic review addressing the quality of health information on the internet. J Gen Intern Med. 2019;34(9):1884-1891. doi:10.1007/s11606-019-05109-0
  33. Ozduran E, Akkoc I, Büyükçoban S, Erkin Y, Hanci V. Readability, reliability and quality of responses generated by ChatGPT, Gemini, and Perplexity for the most frequently asked questions about pain. Medicine (Baltimore). 2025;104(11):e41780. doi:10.1097/MD.0000000000041780
  34. Ömür Arça D, Erdemir İ, Kara F, et al. Assessing the readability, reliability, and quality of Artificial Intelligence chatbot responses to the 100 most searched queries about cardiopulmonary resuscitation: an observational study. Medicine (Baltimore). 2024;103(22):e38352. doi:10.1097/MD.0000000000038352
  35. Aggarwal S, Sridhar J, John VP, et al. An observational study to evaluate readability and reliability of AI-generated brochures for emergency medical conditions. Cureus. 2024;16(8):e68307. doi:10.7759/cureus.68307
  36. Sharma S, Latif Z, Makuvire TT, et al. Readability and accessibility of patient-education materials for heart failure in the United States. J Card Fail. 2025;31(1):154-157. doi:10.1016/j.cardfail.2024.06.015
  37. Ahn AB, Kulhari S, Karimi A, Sundararajan S, Sajatovic M. Readability of patient education material in stroke: a systematic literature review. Top Stroke Rehabil. 2024;31(4):345-360. doi:10.1080/10749357.2023.2259177
  38. Ruan T, Shao X, Sun Y, et al. Evaluation of accuracy, quality, and readability of information on hypothyroidism provided by different Artificial Intelligence chatbot models. Front Public Health. 2025;13: 1698596. doi:10.3389/fpubh.2025.1698596
  39. Büker M, Mercan G. Readability, accuracy, and appropriateness and quality of AI chatbot responses as a patient information source on root canal retreatment: a comparative assessment. Int J Med Inform. 2025; 201:105948. doi:10.1016/j.ijmedinf.2025.105948
  40. Misra AR, Oermann MH, Teague MS, Ledbetter LS. An evaluation of websites offering caregiver education for tracheostomy and home mechanical ventilation. J Pediatr Nurs. 2021;56:64-69. doi:10.1016/j.pedn.2020.09.014
  41. Hanci V, Oner O, Ozduran E, Gokel E. YouTube as a source of information about percutaneous tracheostomy. Gazi Med J. 2023;34:372-378. doi:10.12996/gmj.2023.77
  42. Ozbek IC, Ozduran E, Hanci V. Artificial Intelligence in pain: a comprehensive review. Med J West Black Sea. 2026;10:168-178. doi:10.29058/mjwbs.1881313