SavedAdd your journal

Journals / International Journal of Assessment Tools in Education / 2026 / Volume: 12 Issue: 2

A review of automatic item generation techniques leveraging large language models

Büyük Dil Modelleri Kullanan Otomatik Madde Üretimi Yöntemlerinin İncelenmesi

Bin Tan(University of Alberta)0000-0001-6717-5620
Nour Armoush(University of Alberta)0009-0008-2310-5098
Elisabetta Mazzullo(University of Alberta)0009-0008-4847-9934
Okan Bulut(University of Alberta)0000-0001-5853-1267
Mark Gierl(University of Alberta)0000-0002-2653-1761
Pages
—
Publication Date
2025-06-01
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Therefore, we share two suggestions to enhance the educational foundation for leveraging LLMs in AIG, advocating for interdisciplinary collaborations to exploit the utility and potential of LLMs.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:This study reviews existing research on the use of large language models (LLMs) for automatic item generation (AIG).
📊
Key Findings & Data:We performed a comprehensive literature search across seven research databases, selected studies based on predefined criteria, and summarized 60 relevant studies that employed LLMs in the AIG process.
💡
Conclusion & Impact:Therefore, we share two suggestions to enhance the educational foundation for leveraging LLMs in AIG, advocating for interdisciplinary collaborations to exploit the utility and potential of LLMs.
🔬Design:Review & Systematic Survey
📍Scope:Comprehensive Sample
⚙️Variables:Automatic item generation · Automatic question generation · Large language models

Abstract

This study reviews existing research on the use of large language models (LLMs) for automatic item generation (AIG). We performed a comprehensive literature search across seven research databases, selected studies based on predefined criteria, and summarized 60 relevant studies that employed LLMs in the AIG process. We identified the most commonly used LLMs in current AIG literature, their specific applications in the AIG process, and the characteristics of the generated items. We found that LLMs are flexible and effective in generating various types of items across different languages and subject domains. However, many studies have overlooked the quality of the generated items, indicating a lack of a solid educational foundation. Therefore, we share two suggestions to enhance the educational foundation for leveraging LLMs in AIG, advocating for interdisciplinary collaborations to exploit the utility and potential of LLMs.

Özet

Bu çalışma, otomatik madde üretimi (AIG) için büyük dil modelleri (LLM) kullanımına ilişkin mevcut araştırmaları gözden geçirmektedir. Yedi araştırma veritabanında kapsamlı bir literatür taraması yaptık, önceden tanımlanmış kriterlere göre çalışmalar seçtik ve AIG sürecinde LLM'leri kullanan 60 ilgili çalışmayı özetledik. Mevcut AIG literatüründe en sık kullanılan LLM'leri, AIG sürecindeki özel uygulamalarını ve üretilen maddelerin özelliklerini belirledik. LLM'lerin farklı diller ve konu alanları arasında çeşitli madde türleri üretmede esnek ve etkili olduğunu bulduk. Ancak, birçok çalışma üretilen maddelerin kalitesini göz ardı etti ve bu da sağlam bir eğitim temelinin eksikliğini gösteriyor. Bu nedenle, AIG'de LLM'leri değerlendirmek için eğitim temellerini geliştirmek üzere iki öneri paylaşıyoruz ve LLM'lerin faydasını ve potansiyelini kullanmak için disiplinler arası işbirliklerini savunuyoruz.

🔖

Cite This Article

Tan, B., Armoush, N., Mazzullo, E., Bulut, O., Gierl, M. (2025). A review of automatic item generation techniques leveraging large language models. International Journal of Assessment Tools in Education, Volume: 12 Issue: 2. https://doi.org/10.21449/ijate.1602294

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (104)

  1. Ackerman, R., & Balyan, R. (2023). Automatic multilingual question generation for health data using LLMs. In International Conference on AI-generated Content (pp. 1-11). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-99-7587-7_1
  2. Agrawal, A., & Shukla, P. (2023). Context aware automatic subjective and objective question generation using Fast Text to text transfer learning. International Journal of Advanced Computer Science and Applications, 14(4), 456-463.
  3. Aigo, K., Tsunakawa, T., Nishida, M., & Nishimura, M. (2021). Question generation using knowledge graphs with the T5 language model and masked self-attention. In 2021 IEEE 10th Global Conference on Consumer Electronics (pp. 85 87). IEEE. https://doi.org/10.1109/GCCE53005.2021.9621874
  4. Akyön, F.Ç., Cavusoglu, A.D.E., Cengiz, C., Altinuç, S.O., & Temizel, A. (2022). Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences, 30(5), 1931 1940. https://doi.org/10.55730/1300-0632.3914
  5. Alsubait, T., Parsia, B., & Sattler, U. (2016). Ontology-based multiple choice question generation. KI-Künstliche Intelligenz, 30, 183-188. https://doi.org/10.1007/s13218-015-0405-9
  6. Alves, C.B., Gierl, M.J., & Lai, H. (2010, April). Using automated item generation to promote test design and development [Paper presentation]. American Educational Research Association Annual Meeting, Denver, CO, United States.
  7. Arksey, H., & O'malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19 32. https://doi.org/10.1080/1364557032000119616
  8. Attali, Y., Runge, A., LaFlair, G.T., Yancey, K., Goodwin, S., Park, Y., & Von Davier, A.A. (2022). The interactive reading task: Transformer-based automatic item generation. Frontiers in Artificial Intelligence, 5, 903077. https://doi.org/10.3389/frai.2022.903077
  9. Berger, G., Rischewski, T., Chiruzzo, L., & Rosá, A. (2022). Generation of English question answer exercises from texts using transformers-based models. In 2022 IEEE Latin American Conference on Computational Intelligence (pp. 1-5). IEEE. https://doi.org/10.1109/LA-CCI54402.2022.9981171
  10. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. https://doi.org/10.1080/0969595980050102
  11. Bulathwela, S., Muse, H., & Yilmaz, E. (2023). Scalable educational question generation with pre-trained language models. In International Conference on Artificial Intelligence in Education (pp. 327-339). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-36272-9_27
  12. Bulut, O., & Yildirim-Erbasli, S.N. (2022). Automatic story and item generation for reading comprehension assessments with transformers. International Journal of Assessment Tools in Education, 9(Special Issue), 72-87. https://doi.org/10.21449/ijate.1124382
  13. Bulut, O., Gorgun, G., Yildirim‐Erbasli, S.N., Wongvorachan, T., Daniels, L.M., Gao, Y.,. & Shin, J. (2023). Standing on the shoulders of giants: Online formative assessments as the foundation for predictive learning analytics models. British Journal of Educational Technology, 54(1), 19-39. https://doi.org/10.1111/bjet.13276
  14. Ch, D.R., & Saha, S.K. (2018). Automatic multiple choice question generation from text: A survey. IEEE Transactions on Learning Technologies, 13(1), 14 25. https://doi.org/10.1109/TLT.2018.2889100
  15. Chiang, S.H., Wang, S.C., & Fan, Y.C. (2024). Cdgp: Automatic cloze distractor generation based on pre trained language model. arXiv preprint arXiv:2403.10326. https://doi.org/10.18653/v1/2022.findings-emnlp.429
  16. Chughtai, R., Azam, F., Anwar, M.W., But, W.H., & Farooq, M.U. (2022). A lecture centric automated distractor generation for post-graduate software engineering courses. In 2022 International Conference on Frontiers of Information Technology (FIT) (pp. 100-105). IEEE. https://doi.org/10.1109/FIT57066.2022.00028
  17. Chung, H.L., Chan, Y.H., & Fan, Y.C. (2020). A BERT-based distractor generation scheme with multi tasking and negative answer training strategies. arXiv preprint arXiv:2010.05384. https://arxiv.org/abs/2010.05384
  18. Dalby, D., & Swan, M. (2019). Using digital technology to enhance formative assessment in mathematics classrooms. British Journal of Educational Technology, 50(2), 832-845. https://doi.org/10.1111/bjet.12606
  19. Dembitzer, L., Zelikovitz, S., & Kettler, R.J. (2017). Designing computer-based assessments: Multidisciplinary findings and student perspectives. International Journal of Educational Technology, 4(3), 20 31. https://educationaltechnology.net/ijet/index.php/ijet/article/view/47
  20. Desai, T. (2021). Discourse parsing and its application to question generation [Unpublished dissertation]. The University of Texas at Dallas.
  21. Devlin, J., Chang, M.W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. https://doi.org/10.48550/arXiv.1810.04805
  22. Dijkstra, R., Genç, Z., Kayal, S., & Kamps, J. (2022). Reading comprehension quiz generation using generative pre-trained transformers. In S. Sosnovsky, P. Brusilovsky, & A. Lan (Eds.), Proceedings of the Fourth International Workshop on Intelligent Textbooks 2022 (pp. 4–7). CEUR-WS. http://ceur-ws.org/Vol-3192/
  23. Drori, I., Zhang, S., Shuttleworth, R., Tang, L., Lu, A., Ke, E.,. & Strang, G. (2022). A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. Proceedings of the National Academy of Sciences, 119(32), e2123433119. https://doi.org/10.1073/pnas.2123433119
  24. Falcão, F., Costa, P., & Pêgo, J.M. (2022). Feasibility assurance: a review of automatic item generation in medical assessment. Advances in Health Sciences Education, 27(2), 405-425. https://doi.org/10.1007/s10459-022-10092-z
  25. Femi, J.G., & Nayak, A.K. (2022). EQGTL: An Ensemble Model for Relevant Question Generation using Transfer Learning. In 2022 International Conference on Machine Learning, Computer Systems and Security (pp. 253-258). IEEE. https://doi.org/10.1109/MLCSS57186.2022.00054
  26. Fuadi, M., & Wibawa, A.D. (2022). Automatic question generation from indonesian texts using text-to-text transformers. In 2022 International Conference on Electrical and Information Technology (IEIT) (pp. 84-89). IEEE. https://doi.org/10.1109/IEIT56384.2022.9967858
  27. Fung, Y.C., Kwok, J.C.W., Lee, L.K., Chui, K.T., & U, L.H. (2020). Automatic question generation system for english reading comprehension. In Technology in Education. Innovations for Online Teaching and Learning: 5th International Conference, ICTE 2020, Macau, China, August 19-22, 2020, Revised Selected Papers 5 (pp. 136-146). Springer Singapore. https://doi.org/10.1007/978-981-33-4594-2_12
  28. Fung, Y.C., Lee, L.K., & Chui, K.T. (2023). An automatic question generator for Chinese comprehension. Inventions, 8(1), 31. https://doi.org/10.3390/inventions8010031
  29. Ghanem, B., Coleman, L.L., Dexter, J.R., von der Ohe, S.M., & Fyshe, A. (2022). Question generation for reading comprehension assessment by modeling how and what to ask. arXiv preprint arXiv:2204.02908. https://doi.org/10.48550/arXiv.2204.02908
  30. Gierl, M.J., & Lai, H. (2012). The role of item models in automatic item generation. International Journal of Testing, 12(3), 273 298. https://doi.org/10.1080/15305058.2011.635830
  31. Gierl, M.J., & Lai, H. (2015). Automatic item generation. In Handbook of test development (pp. 410-429). Routledge.
  32. Gierl, M.J., & Lai, H. (2016). A process for reviewing and evaluating generated test items. Educational Measurement: Issues and Practice, 35(4), 6 20. https://doi.org/10.1111/emip.12129
  33. Gierl, M.J., Lai, H., & Tanygin, V. (2021). Advanced methods in automatic item generation. Routledge.
  34. Godslove, J.F., & Nayak, A.K. (2023). Generative model for formulating relevant questions and answers using transfer learning. In AIP Conference Proceedings (Vol. 2819, No. 1). AIP Publishing. https://doi.org/10.1063/5.0136892
  35. Gopal, A. (2022). Automatic question generation for Hindi and Marathi. In 2022 International Conference on Advanced Learning Technologies (ICALT) (pp. 19-21). IEEE. https://doi.org/10.1109/ICALT55010.2022.00012
  36. Goyal, R., Kumar, P., & Singh, V.P. (2023). Automated question and answer generation from texts using text-to-text transformers. Arabian Journal for Science and Engineering, 1-15. https://doi.org/10.1007/s13369-023-07840-7
  37. Granić, A. (2022). Educational technology adoption: A systematic review. Education and Information Technologies, 27(7), 9725-9744. https://doi.org/10.1007/s10639-022-10951-7
  38. Grover, K., Kaur, K., Tiwari, K., Rupali, & Kumar, P. (2021). Deep learning based question generation using t5 transformer. In Advanced Computing: 10th International Conference, IACC 2020, Panaji, Goa, India, December 5–6, 2020, Revised Selected Papers, Part I 10 (pp. 243-255). Springer Singapore. https://doi.org/10.1007/978-981-16-0401-0_18
  39. Han, Z. (2022). Unsupervised multilingual distractor generation for fill-in-the-blank questions [Unpublished thesis]. Uppsala University.
  40. Jiao, Y., Shridhar, K., Cui, P., Zhou, W., & Sachan, M. (2023). Automatic educational question generation with difficulty level controls. In International Conference on Artificial Intelligence in Education (pp. 476-488). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-36272-9_39
  41. Kalpakchi, D., & Boye, J. (2021). BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset. arXiv preprint arXiv:2108.03973. https://doi.org/10.48550/arXiv.2108.03973
  42. Kasakowskij, R., Kasakowskij, T., & Seidel, N. (2022). Generation of multiple true false questions. 20. Fachtagung Bildungstechnologien. https://doi.org/10.18420/delfi2022-026
  43. Khandait, K., Bhura, S., & Asole, S.S. (2022). Automatic question generation through word vector synchronization using lamma. Indian Journal of Computer Science and Engineering, 13(4), 1083-1095. https://doi.org/10.21817/indjcse/2022/v13i4/221304046
  44. Kosh, A.E., Simpson, M.A., Bickel, L., Kellogg, M., & Sanford‐Moore, E. (2019). A cost–benefit analysis of automatic item generation. Educational Measurement: Issues and Practice, 38(1), 48-53. https://doi.org/10.1111/emip.12237
  45. Kumar, A., Kharadi, A., Singh, D., & Kumari, M. (2021). Automatic question-answer pair generation using deep learning. In 2021 Third International Conference on Inventive Research in Computing Applications (pp. 794 799). IEEE. https://doi.org/10.1109/ICIRCA51532.2021.9544654
  46. Kumar, N.S., Mali, R., Ratnam, A., Kurpad, V., & Magapu, H. (2022). Identification and addressal of knowledge gaps in students. In 2022 3rd International Conference for Emerging Technology (pp. 1-6). IEEE. https://doi.org/10.1109/INCET54531.2022.9824483
  47. Kumar, S., Chauhan, A., & Kumar C, P. (2022). Learning enhancement using question-answer generation for e-book using contrastive fine-tuned T5. In International Conference on Big Data Analytics (pp. 68 87). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-24094-2_5
  48. Kumari, V., Keshari, S., Sharma, Y., & Goel, L. (2022). Context-based question answering system with suggested questions. In 2022 12th International Conference on Cloud Computing, Data Science & Engineering (pp. 368 373). IEEE. https://doi.org/10.1109/Confluence52989.2022.9734207
  49. Kuo, C.Y., & Wu, H.K. (2013). Toward an integrated model for designing assessment systems: An analysis of the current status of computer-based assessments in science. Computers & Education, 68, 388-403. https://doi.org/10.1016/j.compedu.2013.06.002
  50. Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S. (2020). A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30, 121-204. https://doi.org/10.1007/s40593-019-00186-y
  51. Lai, H., Alves, C., & Gierl, M.J. (2009). Using automatic item generation to address item demands for CAT. In D.J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing. www.psych.umn.edu/psylabs/CATCentral/
  52. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. https://doi.org/10.48550/arXiv.1909.11942
  53. Lee, H., Chung, H.Q., Zhang, Y., Abedi, J., & Warschauer, M. (2020). The effectiveness and features of formative assessment in US K-12 education: A systematic review. Applied Measurement in Education, 33(2), 124 140. https://doi.org/10.1080/08957347.2020.1732383
  54. Lim, Y.S. (2019). Students’ perception of formative assessment as an instructional tool in medical education. Medical Science Educator, 29(1), 255 263. https://doi.org/10.1007/s40670-018-00687-w
  55. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D.,. & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. https://doi.org/10.48550/arXiv.1907.11692
  56. Lu, O.H.T., Huang, A.Y.Q., Tsai, D.C.L., & Yang, S.J.H. (2021). Expert-authored and machine-generated short- answer questions for assessing students’ learning performance. Educational Technology & Society, 24(3), 159–173. https://www.jstor.org/stable/27032863
  57. Maheen, F., Asif, M., Ahmad, H., Ahmad, S., Alturise, F., Asiry, O., & Ghadi, Y.Y. (2022). Automatic computer science domain multiple-choice questions generation based on informative sentences. PeerJ Computer Science, 8, e1010. https://doi.org/10.7717/peerj-cs.1010
  58. Malhar, A., Sawant, P., Chhadva, Y., & Kurhade, S. (2022). Deep learning-based Answering Questions using T5 and Structured Question Generation System. In 2022 6th International Conference on Intelligent Computing and Control Systems (pp. 1544-1549). IEEE. https://doi.org/10.1109/ICICCS53718.2022.9788264
  59. Mathur, A., & Suchithra, M. (2022). Application of abstractive summarization in multiple choice question generation. In 2022 International Conference on Computational Intelligence and Sustainable Engineering Solutions (pp. 409 413). IEEE. https://doi.org/10.1109/CISES54857.2022.9844396
  60. Matsumori, S., Okuoka, K., Shibata, R., Inoue, M., Fukuchi, Y., & Imai, M. (2023). Mask and cloze: Automatic open cloze question generation using a masked language model. IEEE Access, 11, 9835-9850. https://doi.org/10.1109/ACCESS.2023.3239005
  61. Maurya, K.K., & Desarkar, M.S. (2020). Learning to distract: A hierarchical multi-decoder network for automated generation of long distractors for multiple-choice questions for reading comprehension. In Proceedings of the 29th ACM international conference on information & knowledge management (pp. 1115 1124). https://doi.org/10.1145/3340531.3411997
  62. Mazzullo, E., Bulut, O., Wongvorachan, T., & Tan, B. (2023). Learning Analytics in the Era of Large Language Models. Analytics, 2(4), 877 898. https://doi.org/10.3390/analytics2040046
  63. Min, B., Ross, H., Sulem, E., Veyseh, A.P.B., Nguyen, T.H., Sainz, O.,. & Roth, D. (2023). Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys, 56(2), 1-40. https://doi.org/10.1145/3605943
  64. Muse, H., Bulathwela, S., & Yilmaz, E. (2022). Pre-training with scientific text improves educational question generation. arXiv preprint arXiv:2212.03869. https://doi.org/10.48550/arXiv.2212.03869
  65. Newton, P.E. (2007). Clarifying the purposes of educational assessment. Assessment in education, 14(2), 149-170. https://doi.org/10.1080/09695940701478321
  66. Nguyen, H.A., Bhat, S., Moore, S., Bier, N., & Stamper, J. (2022). Towards generalized methods for automatic question generation in educational domains. In European conference on technology enhanced learning (pp. 272-284). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-16290-9_20
  67. Nittala, S., Agarwal, P., Vishnu, R., & Shanbhag, S. (2022). Speaker Diarization and BERT-Based Model for Question Set Generation from Video Lectures. In Information and Communication Technology for Competitive Strategies ICT: Applications and Social Interfaces (pp. 441 452). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-19-0095-2_42
  68. Offerijns, J., Verberne, S., & Verhoef, T. (2020). Better distractions: Transformer-based distractor generation and multiple-choice question filtering. arXiv preprint arXiv:2010.09598. https://doi.org/10.48550/arXiv.2010.09598
  69. Pochiraju, D., Chakilam, A., Betham, P., Chimulla, P., & Rao, S.G. (2023). Extractive summarization and multiple-choice question generation using XLNet. In 2023 7th International Conference on Intelligent Computing and Control Systems (pp. 1001-1005). IEEE. https://doi.org/10.1109/ICICCS56967.2023.10142220
  70. Pugh, D., De Champlain, A., Gierl, M., Lai, H., & Touchie, C. (2020). Can automated item generation be used to develop high quality MCQs that assess application of knowledge?. Research and Practice in Technology Enhanced Learning, 15, 1 13. https://doi.org/10.1186/s41039-020-00134-8
  71. Qiu, X., Xue, H., Liang, L., Xie, Z., Liao, S., & Shi, G. (2021). Automatic generation of multiple-choice cloze-test questions for lao language learning. In 2021 International Conference on Asian Language Processing (pp. 125 130). IEEE. https://doi.org/10.1109/IALP54817.2021.9675153
  72. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M.,. & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140), 1-67. https://doi.org/10.48550/arXiv.1910.10683
  73. Raina, V., & Gales, M. (2022). Multiple-choice question generation: Towards an automated assessment framework. arXiv preprint arXiv:2209.11830. https://doi.org/10.48550/arXiv.2209.11830
  74. Ratcheva, M.G., Navale, R., & Desai, B.C. (2022). An online MCQ sub-system for CrsMgr. In Proceedings of the 26th International Database Engineered Applications Symposium (pp. 128-133). https://doi.org/10.1145/3548785.3548789
  75. Rodriguez-Torrealba, R., Garcia-Lopez, E., & Garcia-Cabot, A. (2022). End-to-end generation of multiple-xhoice questions using text-to-text transfer transformer models. Expert Systems with Applications, 208, 118258. https://doi.org/10.1016/j.eswa.2022.118258
  76. Rudner, L.M. (2009). Implementing the graduate management admission test computerized adaptive test. In Elements of adaptive testing (pp. 151-165). Springer New York. https://doi.org/10.1007/978-0-387-85461-8_8
  77. Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. https://doi.org/10.48550/arXiv.1910.01108
  78. Sayin, A., & Gierl, M. (2024). Using OpenAI GPT to generate reading comprehension items. Educational Measurement: Issues and Practice, 43(1), 5 18. https://doi.org/10.1111/emip.12590
  79. Shan, J., Nishihara, Y., Maeda, A., & Yamanishi, R. (2022). Question generation for reading comprehension test complying with types of question. Journal of Information Science & Engineering, 38(3). https://doi.org/10.6688/JISE.202205_38(3).0005
  80. Shan, J., Nishihara, Y., Yamanishi, R., & Maeda, A. (2019). Question generation for reading comprehension of language learning test: A method using Seq2Seq approach with transformer model. In 2019 International Conference on Technologies and Applications of Artificial Intelligence (pp. 1-6). IEEE. https://doi.org/10.1109/TAAI48200.2019.8959903
  81. Shridhar, K., Macina, J., El-Assady, M., Sinha, T., Kapur, M., & Sachan, M. (2022). Automatic generation of socratic subquestions for teaching math word problems. arXiv preprint arXiv:2211.12835. https://doi.org/10.48550/arXiv.2211.12835
  82. Singh, J., McCann, B., Socher, R., & Xiong, C. (2019). BERT is not an interlingua and the bias of tokenization. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019) (pp. 47-55). https://doi.org/10.18653/v1/D19-6106
  83. Spector, J.M., Ifenthaler, D., Samspon, D., Yang, L., Mukama, E., Warusavitarana, A., … Gibson, D.C. (2016). Technology enhanced formative assessment for 21st century learning. Educational Technology & Society, 19(3), 58 71. https://www.jstor.org/stable/jeductechsoci.19.3.58
  84. Srivastava, M., & Goodman, N. (2021). Question generation for adaptive education. arXiv preprint arXiv:2106.04262. https://doi.org/10.48550/arXiv.2106.04262
  85. Steuer, T., Filighera, A., & Rensing, C. (2020). Exploring artificial jabbering for automatic text comprehension question generation. In Addressing Global Challenges and Quality Education: 15th European Conference on Technology Enhanced Learning, EC-TEL 2020, Heidelberg, Germany, September 14–18, 2020, Proceedings 15 (pp. 1-14). Springer International Publishing. https://doi.org/10.1007/978-3-030-57717-9_1
  86. Tsai, D.C., Chang, W., & Yang, S. (2021). Short answer questions generation by Fine-Tuning BERT and GPT-2. In Proceedings of the 29th International Conference on Computers in Education Conference (Vol. 64). https://icce2021.apsce.net/wp-content/uploads/2021/12/ICCE2021-Vol.II-PP.-508-514.pdf
  87. von Davier, M. (2019). Training Optimus prime, MD: Generating medical certification items by fine-tuning OpenAI's gpt2 transformer model. arXiv preprint arXiv:1908.08594. https://doi.org/10.48550/arXiv.1908.08594
  88. Vu, N., & Van Nguyen, K. (2022). Enhancing Vietnamese question generation with reinforcement learning. In Asian Conference on Intelligent Information and Database Systems (pp. 559 570). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-21743-2_45
  89. Wang, B., Yao, T., Chen, W., Xu, J., & Wang, X. (2021). Multi-lingual question generation with language agnostic language model. In Findings of the Association for Computational Linguistics: ACL IJCNLP 2021 (pp. 2262 2272). https://aclanthology.org/2021.findings-acl.199.pdf
  90. Wang, H.C., Maslim, M., & Kan, C.H. (2023). A question–answer generation system for an asynchronous distance learning platform. Education and Information Technologies, 28(9), 12059-12088. https://doi.org/10.1007/s10639-023-11675-y
  91. Wang, Z., Valdez, J., Basu Mallick, D., & Baraniuk, R.G. (2022). Towards human-like educational question generation with large language models. In International conference on artificial intelligence in education (pp. 153-166). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-11644-5_13
  92. Wylie, E.C., & Lyon, C.J. (2015). The fidelity of formative assessment implementation: Issues of breadth and quality. Assessment in Education: Principles, Policy & Practice, 22(1), 140-160. https://doi.org/10.1080/0969594X.2014.990416
  93. Xie, J., Peng, N., Cai, Y., Wang, T., & Huang, Q. (2021). Diverse distractor generation for constructing high-quality multiple choice questions. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30, 280 291. https://doi.org/10.1109/TASLP.2021.3138706
  94. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., & Le, Q.V. (2019). Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32. https://dl.acm.org/doi/10.5555/3454287.3454804
  95. Yen, Y.-C., Ho, R.-G., Liao, W.-W., & Chen, L.-J. (2012). Reducing the impact of inappropriate items on reviewable computerized adaptive testing. Educational Technology & Society, 15(2), 231–243. https://www.jstor.org/stable/jeductechsoci.15.2.231
  96. Zhang, C. (2023). Automatic Generation of Multiple-Choice Questions. arXiv preprint arXiv: 2303.14576v1. https://doi.org/10.48550/arXiv.2303.14576
  97. Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y.,. & Wen, J.R. (2023). A survey of large language models. arXiv preprint arXiv:2303.18223. https://doi.org/10.48550/arXiv.2303.18223
  98. Zhao, Z., Hou, Y., Wang, D., Yu, M., Liu, C., & Ma, X. (2022). Educational question generation of children storybooks via question type distribution learning and event-centric summarization. arXiv preprint arXiv:2203.14187. https://doi.org/10.48550/arXiv.2203.14187
  99. Bulut, O., Gorgun, G., Yildirim‐Erbasli, S.N., Wongvorachan, T., Daniels, L.M., Gao, Y.,. & Shin, J. (2023). Standing on the shoulders of giants: Online formative assessments as the foundation for predictive learning analytics models. British Journal of Educational Technology, 54(1), 19-39. https://doi.org/10.1111/bjet.13276
  100. Drori, I., Zhang, S., Shuttleworth, R., Tang, L., Lu, A., Ke, E.,. & Strang, G. (2022). A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. Proceedings of the National Academy of Sciences, 119(32), e2123433119. https://doi.org/10.1073/pnas.2123433119
  101. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D.,. & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. https://doi.org/10.48550/arXiv.1907.11692
  102. Min, B., Ross, H., Sulem, E., Veyseh, A.P.B., Nguyen, T.H., Sainz, O.,. & Roth, D. (2023). Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys, 56(2), 1-40. https://doi.org/10.1145/3605943
  103. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M.,. & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140), 1-67. https://doi.org/10.48550/arXiv.1910.10683
  104. Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y.,. & Wen, J.R. (2023). A survey of large language models. arXiv preprint arXiv:2303.18223. https://doi.org/10.48550/arXiv.2303.18223