SavedAdd your journal

Journals / Scientific Journal of Mehmet Akif Ersoy University / 2026 / Volume: 9 Issue: 1

Towards Benchmarking Transformer Models for Biomedical Text Simplification

Öykü Berfin Mercan(IZMIR KATIP CELEBI UNIVERSITY)0000-0001-7727-0197
Mansur Alp Toçoğlu(IZMIR KATIP CELEBI UNIVERSITY)0000-0003-1784-9003
Nezihe Turhan Turan(IZMIR KATIP CELEBI UNIVERSITY)0000-0002-9012-4386
Aytuğ Onan(IZMIR INSTITUTE OF TECHNOLOGY)0000-0002-9434-5880
Pages
—
Publication Date
2026-04-15
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The results indicate that BART based models achieve superior performance in the medical text simplification task.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Biomedical texts typically contain a high level of technical terminology and complex sentence structures, which limits their comprehensibility for readers without domain expertise.
📊
Key Findings & Data:In this context, this study investigates the text simplification performance of pre-trained general-purpose and domain-specific language models (PLMs) for biomedical texts.
💡
Conclusion & Impact:The results indicate that BART based models achieve superior performance in the medical text simplification task.
🔬Design:Review & Systematic Survey
📍Scope:Comprehensive Sample
⚙️Variables:Text simplification · Pre-trained language models · Biomedical text

Abstract

Biomedical texts typically contain a high level of technical terminology and complex sentence structures, which limits their comprehensibility for readers without domain expertise. Text simplification, a natural language processing problem, aims to transform complex texts into a more readable and accessible form while preserving their original semantic content. Especially in biomedical texts, simplification can play an essential role in making scientific information understandable to patients and the general public. In this context, this study investigates the text simplification performance of pre-trained general-purpose and domain-specific language models (PLMs) for biomedical texts. The experiments utilize the Cochrane-Simplification dataset, which comprises technical abstracts from systematic reviews and their corresponding plain language summaries. General-purpose models and summarization tuned variants (BART-Large, BART-Large-CNN, BART-Large-XSum, PEGASUS-Large, PEGASUS-XSum, T5 and FLAN-T5) are compared alongside domain-specific models (BioBARTv2-Large, SciFive, Clinical-T5) under comparable fine-tuning settings. The models were compared using ROUGE, BLEU, BERTScore and SARI metrics to measure textual similarity and semantic coherence. The results indicate that BART based models achieve superior performance in the medical text simplification task.

🔖

Cite This Article

Mercan, Ö. B., Toçoğlu, M. A., Turan, N. T., Onan, A. (2026). Towards Benchmarking Transformer Models for Biomedical Text Simplification. Scientific Journal of Mehmet Akif Ersoy University, Volume: 9 Issue: 1. https://doi.org/10.70030/sjmakeu.1879926

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (47)

  1. Swanson, K., He, S., Calvano, J., Chen, D., Telvizian, T., Jiang, L.,. & Lee, J. (2024). Biomedical text readability after hypernym substitution with fine-tuned large language models. PLOS Digital Health, 3(4), e0000489.
  2. Moramarco, F., Juric, D., Savkov, A., Flann, J., Lehl, M., Boda, K.,. & Hammerla, N. (2022, February). Towards more patient friendly clinical notes through language models and ontologies. In AMIA Annual Symposium Proceedings (Vol. 2021, p. 881).
  3. Gotlieb, R., Praska, C., Hendrickson, M. A., Marmet, J., Charpentier, V., Hause, E.,. & Pitt, M. B. (2022). Accuracy in patient understanding of common medical phrases. JAMA network open, 5(11), e2242972-e2242972.
  4. Paulson, D., & Hernandez, L. (2025). Evaluating Language Models for Simplifying Health Literacy Materials.
  5. Bakker, J., Papandreou-Lazos, T., & Kamps, J. (2024, November). Biomedical text simplification models trained on aligned abstracts and lay summaries. In The Thirty-Third Text REtrieval Conference Proceedings (TREC 2024), Gaithersburg, MD, USA (Vol. 1329).
  6. Li, Z., Belkadi, S., Micheletti, N., Han, L., Shardlow, M., & Nenadic, G. (2024, June). Investigating large language models and control mechanisms to improve text readability of biomedical abstracts. In 2024 IEEE 12th International Conference on Healthcare Informatics (ICHI) (pp. 265-274). IEEE.
  7. Cripwell, L., Legrand, J., & Gardent, C. (2023, May). Document-level planning for text simplification. In 17th Conference of the European Chapter of the Association for Computational Linguistics (pp. 993-1006). Association for Computational Linguistics.
  8. Rahman, M. M., Irbaz, M. S., North, K., Williams, M. S., Zampieri, M., & Lybarger, K. (2024). Health text simplification: An annotated corpus for digestive cancer education and novel strategies for reinforcement learning. Journal of Biomedical Informatics, 158, 104727.
  9. Shardlow, Matthew. (2014). A Survey of Automated Text Simplification. International Journal of Advanced Computer Science and Applications. 4. 10.14569/SpecialIssue.2014.040109.
  10. Vora, A., Chaudhari, T., Hotha, S., & Sonawane, S. (2025). S-3 Pipeline for Biomedical Text Simplification.
  11. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N.,. & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
  12. Celikten, T., & Onan, A. (2025). Benchmarking Large Language Models for Biomedical Literature Summarization: Abstractive vs. Extractive Paradigms. IEEE Access.
  13. Xie, Q., Bishop, J. A., Tiwari, P., & Ananiadou, S. (2022). Pre-trained language models with domain knowledge for biomedical extractive summarization. Knowledge-Based Systems, 252, 109460.
  14. Daraghmi, E., Atwe, L., & Jaber, A. (2025). A comparative study of pegasus, bart, and t5 for text summarization across diverse datasets. Future Internet, 17(9), 389.
  15. Artsi, Y., Sorin, V., Konen, E., Glicksberg, B. S., Nadkarni, G., & Klang, E. (2024). Large language models in simplifying radiological reports: systematic review. medRxiv, 2024-01.
  16. Picton, B., Andalib, S., Spina, A., Camp, B., Solomon, S. S., Liang, J.,. & Oh, M. Y. (2025). Assessing AI simplification of medical texts: readability and content fidelity. International Journal of Medical Informatics, 195, 105743.
  17. Devaraj, A., Marshall, I., Wallace, B. C., & Li, J. J. (2021, June). Paragraph-level simplification of medical texts. In Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies (pp. 4972-4984).
  18. Pattisapu, N., Prabhu, N., Bhati, S., & Varma, V. (2020, July). Leveraging social media for medical text simplification. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 851-860).
  19. Makhmutova, L., Salton, G. D., Perez-Tellez, F., & Ross, R. J. (2024). Automated Medical Text Simplification for Enhanced Patient Access. In BIOSTEC (2) (pp. 208-218).
  20. Ferreira, D. J. B., Almeida, T., & Matos, S. (2025, July). A Framework for Fine-Grained Complexity Control in Health Answer Generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop) (pp. 1111-1131).
  21. Rahulprasath, S., Harshan, P., Kabilash, P. V., Lakshithraj, A., & Sreemathy, J. (2025, June). AI in Healthcare: Simplifying Medical Reports for Enhanced Patient Comprehension. In 2025 International Conference on Emerging Technologies in Computing and Communication (ETCC) (pp. 1-6). IEEE.
  22. GEM. (n.d.). cochrane-simplification [Data set]. Hugging Face. Retrieved November, 2025, from https://huggingface.co/datasets/GEM/cochrane-simplification
  23. Yogarajan, V., Gouk, H., Smith, T., Mayo, M., & Pfahringer, B. (2020, March). Comparing high dimensional word embeddings trained on medical text to bag-of-words for predicting medical codes. In Asian Conference on Intelligent Information and Database Systems (pp. 97-108). Cham: Springer International Publishing.
  24. Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O.,. & Zettlemoyer, L. (2020, July). BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th annual meeting of the association for computational linguistics (pp. 7871-7880).
  25. Zhang, J., Zhao, Y., Saleh, M., & Liu, P. (2020, November). Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In International conference on machine learning (pp. 11328-11339). PMLR.
  26. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M.,. & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140), 1-67.
  27. Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W.,. & Wei, J. (2024). Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70), 1-53.
  28. Yuan, H., Yuan, Z., Gan, R., Zhang, J., Xie, Y., & Yu, S. (2022). BioBART: Pretraining and evaluation of a biomedical generative language model. arXiv preprint arXiv:2204.03905.
  29. Phan, L. N., Anibal, J. T., Tran, H., Chanana, S., Bahadroglu, E., Peltekian, A., & Altan-Bonnet, G. (2021). Scifive: a text-to-text transformer model for biomedical literature. arXiv preprint arXiv:2106.03598.
  30. Lehman, E., & Johnson, A. (2023). Clinical-t5: Large language models built using mimic clinical text. PhysioNet, 101, 215-220.
  31. Goyal, T., Xu, J., Li, J. J., & Durrett, G. (2022, May). Training dynamics for text summarization models. In Findings of the Association for Computational Linguistics: ACL 2022 (pp. 2061-2073).
  32. Zhang, Y., Ni, A., Yu, T., Zhang, R., Zhu, C., Deb, B.,. & Radev, D. (2021). An exploratory study on long dialogue summarization: What works and what's next. arXiv preprint arXiv:2109.04609.
  33. Lin, C. Y. (2004, July). Rouge: A package for automatic evaluation of summaries. In Text summarization branches out (pp. 74-81).
  34. Zaman, F., Kamiran, F., Shardlow, M., Hassan, S. U., Karim, A., & Aljohani, N. R. (2024). SATS: simplification aware text summarization of scientific documents. Frontiers in Artificial Intelligence, 7, 1375419.
  35. Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002, July). Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics (pp. 311-318).
  36. Ormaechea, L., & Tsourakis, N. (2024). Automatic text simplification for French: model fine-tuning for simplicity assessment and simpler text generation. International Journal of Speech Technology, 27(4), 957-976.
  37. Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019). Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675.
  38. Xu, W., Napoles, C., Pavlick, E., Chen, Q., & Callison-Burch, C. (2016). Optimizing statistical machine translation for text simplification. Transactions of the Association for Computational Linguistics, 4, 401-415.
  39. Swanson, K., He, S., Calvano, J., Chen, D., Telvizian, T., Jiang, L.,. & Lee, J. (2024). Biomedical text readability after hypernym substitution with fine-tuned large language models. PLOS Digital Health, 3(4), e0000489.
  40. Moramarco, F., Juric, D., Savkov, A., Flann, J., Lehl, M., Boda, K.,. & Hammerla, N. (2022, February). Towards more patient friendly clinical notes through language models and ontologies. In AMIA Annual Symposium Proceedings (Vol. 2021, p. 881).
  41. Gotlieb, R., Praska, C., Hendrickson, M. A., Marmet, J., Charpentier, V., Hause, E.,. & Pitt, M. B. (2022). Accuracy in patient understanding of common medical phrases. JAMA network open, 5(11), e2242972-e2242972.
  42. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N.,. & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
  43. Picton, B., Andalib, S., Spina, A., Camp, B., Solomon, S. S., Liang, J.,. & Oh, M. Y. (2025). Assessing AI simplification of medical texts: readability and content fidelity. International Journal of Medical Informatics, 195, 105743.
  44. Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O.,. & Zettlemoyer, L. (2020, July). BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th annual meeting of the association for computational linguistics (pp. 7871-7880).
  45. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M.,. & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140), 1-67.
  46. Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W.,. & Wei, J. (2024). Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70), 1-53.
  47. Zhang, Y., Ni, A., Yu, T., Zhang, R., Zhu, C., Deb, B.,. & Radev, D. (2021). An exploratory study on long dialogue summarization: What works and what's next. arXiv preprint arXiv:2109.04609.