SavedAdd your journal

Journals / Nevşehir Hacı Bektaş Veli Üniversitesi SBE Dergisi / 2026 / Sayı: YZ (AI) ve Gelecek Perspektifler Özel Sayı

Evaluating foreign language speaking proficiency using generative artificial intelligence

Yabancı dilde konuşma becerisinin üretken yapay zekâ ile değerlendirilmesi

Oğuz İbrahim Dündar(ANADOLU ÜNİVERSİTESİ)0000-0002-9574-6125
Pages
169–184
Publication Date
2025-12-07
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The study emphasizes the importance of adopting GAI-based assessment practices that align with pedagogical goals and are ethically and culturally responsive.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:This rapid review study explores the role of generative artificial intelligence (GAI)-powered tools in assessing the speaking skills of foreign language learners, employing a thematic analysis approach.
📊
Key Findings & Data:A total of 67 studies focused on the topic were initially identified through a literature search.
💡
Conclusion & Impact:The study emphasizes the importance of adopting GAI-based assessment practices that align with pedagogical goals and are ethically and culturally responsive.
🔬Design:Review & Systematic Survey
📍Scope:2018 Verileri
⚙️Variables:Hızlı derleme · Konuşma becerisi · Ölçme-değerlendirme

Abstract

This rapid review study explores the role of generative artificial intelligence (GAI)-powered tools in assessing the speaking skills of foreign language learners, employing a thematic analysis approach. A total of 67 studies focused on the topic were initially identified through a literature search. Of these, seven empirical studies, published between 2018 and 2025, involving foreign language learners, and utilizing GAI tools, were selected for in-depth analysis based on defined inclusion criteria. Data collection was carried out using four major academic databases: Scopus, Google Scholar, Web of Science, and EBSCO. The findings are organized under four main themes: (1) automated feedback and personalized assessment processes, (2) linguistic competence and learner autonomy, (3) innovative technical and pedagogical approaches, and (4) ethical, cultural, and systemic limitations. Large language models and automatic speech recognition technologies provide learners with immediate, detailed, and targeted feedback, thereby personalizing learning and significantly reducing teachers’ assessment workload. However, these systems still face limitations in terms of cultural sensitivity, emotional intelligence, and contextual adaptability. The study emphasizes the importance of adopting GAI-based assessment practices that align with pedagogical goals and are ethically and culturally responsive.

Özet

Bu hızlı derleme çalışması, üretken yapay zekâ (ÜYZ) destekli araçların yabancı dil öğrenen bireylerin konuşma becerilerinin değerlendirilmesindeki rolünü tematik analiz yöntemiyle incelemektedir. Çalışma kapsamında, alan yazında konuya odaklanan 67 çalışma taranmış; 2018-2025 yılları arasında yayımlanan, ÜYZ araçlarının kullanıldığı ve yabancı dil öğrenenlerle yürütülmüş 7 ampirik araştırma belirlenen ölçütlere göre derinlemesine analiz edilmek üzere seçilmiştir. Veri toplama sürecinde Scopus, Google Scholar, Web of Science ve EBSCO gibi dört büyük akademik veri tabanı kullanılmıştır. Elde edilen bulgular dört ana tema etrafında toplanmaktadır: (1) otomatik geribildirim ve bireyselleştirilmiş değerlendirme (2) dil yeterlilikleri ve öğrenen özerkliği üzerindeki etkiler, (3) yenilikçi teknik yaklaşımlar, (4) sistemsel sınırlılıklar ve etik meseleler. Büyük dil modelleri ve otomatik konuşma tanıma teknolojileri, öğrencilere anlık, detaylı ve hedefe yönelik geribildirim sunarak öğrenme süreçlerini bireyselleştirmekte ve öğretmenlerin ölçme-değerlendirme yükünü önemli ölçüde azaltmaktadır. Ancak bu sistemler, kültürel duyarlılık, duygusal zekâ ve bağlamsal esneklik açısından hâlâ çeşitli sınırlılıklar taşımaktadır. Çalışma, ÜYZ tabanlı değerlendirme uygulamalarının pedagojik hedeflerle uyumlu, etik ve kültürel açıdan duyarlı bir yaklaşımla ele alınmasının gerekliliğini vurgulamaktadır.

🔖

Cite This Article

Dündar, O. İ. (2025). Evaluating foreign language speaking proficiency using generative artificial intelligence. Nevşehir Hacı Bektaş Veli Üniversitesi SBE Dergisi, Sayı: YZ (AI) ve Gelecek Perspektifler Özel Sayı, 169–184. https://doi.org/10.30783/nevsosbilen.1746385

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (55)

  1. Ayeni, O. O., Al Hamad, N. M., Chisom, O. N., Osawaru, B., & Adewusi, O. E. (2024). AI in education: A review of personalized learning and educational technology. GSC Advanced Research and Reviews, 18(2), 261-271. https://doi.org/10.30574/gscarr.2024.18.2.0062
  2. Bae, M., Wang, J., Xue, H., Chong, S. M., Kwon, O., & Ki, C. W. (2024). Does ChatGPT help or hinder education? Exploring its benefits, challenges, student guilt, and the need for educator training. International Journal of Fashion Design, Technology and Education, 1-16. https://doi.org/10.1080/17543266.2024.2430585
  3. Bolton, C., Machová, V., Kovacova, M., & Valaskova, K. (2018). The power of human–machine collaboration: Artificial intelligence, business automation, and the smart economy. Economics, Management, and Financial Markets, 13(4), 51-56.
  4. Celik, B., Yildiz, Y., & Kara, S. (2025). Using ChatGPT as a virtual speaking tutor to boost EFL learners’ speaking self-efficacy. Australian Journal of Applied Linguistics, 8(1).https://doi.org/10.29140/ajal.v8n1.102418
  5. Dai, M. (2022). Intelligent Correction System of Students’ English Pronunciation Errors Based on Speech Recognition Technology. Journal of Information & Knowledge Management, 21(Supp02), 2240013. https://doi.org/10.1142/S0219649222400135
  6. Derner, E., & Batistič, K. (2023). Beyond the safeguards: Exploring the security risks of chatgpt. arXiv preprint https://doi.org/10.48550/arXiv.2305.08005
  7. Dobbins, M. (2017). Rapid review guidebook. Natl Collab Cent Method Tools, 13, 25.
  8. Dündar, A. E. (2024). Création et évaluation de l'efficacité d'un système tutoriel intelligent pour l'enseignement de la grammaire du FLE. (Yayın No. 914033) [Doktora tezi, Anadolu Üniversitesi]. YÖK Ulusal Tez Merkezi.
  9. Dündar, O. İ. (2024). Utilisation Potentielle De Chatgpt Dans L'apprentissage Des Langues Etrangères: Exploration Des Possibilités Selon Les Niveaux Langagiers Du CECRL. Kahramanmaraş Sütçü İmam Üniversitesi Sosyal Bilimler Dergisi, 21(1), 63-75. https://doi.org/10.33437/ksusbd.1384040
  10. Dündar, O. İ., & Aydoğu, C. (2024). Création d'un Agent Conversationnel Pour Une Classe De Françaıs Langue Etrangère: Les Effets Sur L’autonomıe De L’apprenant. Çukurova Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 33(2), 750-762. https://doi.org/10.35379/cusosbil.1411764
  11. Dündar, O. İ., & Aydoğu, C. (2024). Rencontrer l’Autre de la perspective actionnelle via l’intelligence artificielle. Balıkesir Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 27(51), 1-16. https://doi.org/10.31795/baunsobed.1415369
  12. Gaballo, V. (2024, November). Revolutionizing language teaching: AI in oral language assessment. In Conference Proceedings. Innovation in Language Learning 2024.
  13. Gammoh, L. A. (2024). ChatGPT in academia: Exploring university students’ risks, misuses, and challenges in Jordan. Journal of Further and Higher Education, 48(6), 608-624. https://doi.org/10.1080/0309877X.2024.2378298
  14. Görgüt, R. Ç. (2024). Yapay zekâ ve matematik eğitimi. Eğitim ve Bilim 2023-IV, 43.
  15. Halagatti, M., Gadag, S., Mahantshetti, S., Hiremath, C. V., Tharkude, D., & Banakar, V. (2023). Artificial intelligence: the new tool of disruption in educational performance assessment. In Smart analytics, artificial intelligence and sustainable performance management in a global digitalised economy (pp. 261-287). Emerald Publishing Limited. https://doi.org/10.1108/S1569-37592023000110A014
  16. Hashim, S., Omar, M. K., Ab Jalil, H., & Sharef, N. M. (2022). Trends on technologies and artificial intelligence in education for personalized learning: systematic literature. Journal of Academic Research in Progressive Education and Development, 12(1), 884-903. http://dx.doi.org/10.6007/IJARPED/v11-i1/12230
  17. Hawanti, S., & Zubaydulloevna, K. M. (2023). AI chatbot-based learning: alleviating students' anxiety in English writing classroom. Bulletin of Social Informatics Theory and Application, 7(2), 182-192. https://doi.org/10.31763/businta.v7i2.659
  18. Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S. (2022). Artificial intelligence for assessment and feedback to enhance student success in higher education. Mathematical Problems in Engineering, 2022(1). https://doi.org/10.1155/2022/5215722
  19. İnal, E. (2020). Yabancı dil öğretiminde konuşma becerisinin geliştirilmesi ve ölçme değerlendirme süreçlerine yönelik düşünceler: öğretici yeterlikleri. Aydın TÖMER Dil Dergisi, 5(2), 189-204. https://doi.org/10.17932/IAU.TOMER.2016.019/tomer_v05i2003
  20. Jeon, J., Lee, S., & Choe, H. (2023). Beyond ChatGPT: A conceptual framework and systematic review of speech-recognition chatbots for language learning. Computers & Education, 206. https://doi.org/10.1016/j.compedu.2023.104898
  21. Khatri, B. B., & Karki, P. D. (2023). Artificial intelligence (AI) in higher education: Growing academic integrity and ethical concerns. Nepalese Journal of Development and Rural Studies, 20(01), 1-7. https://doi.org/10.3126/njdrs.v20i01.64134
  22. Kopparapu, S. K., Bhat, C., & Panda, A. (2024). Spoken grammar assessment using LLM. arXiv preprint https://doi.org/10.48550/arXiv.2410.01579
  23. Kumar, S., Musharaf, D., Musharaf, S., & Sagar, A. K. (2023, June). A comprehensive review of the latest advancements in large generative AI models. In International Conference on Advanced Communication and Intelligent Systems (pp. 90-103). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-45121-8_9
  24. Lu, Y. (2019). Artificial intelligence: a survey on evolution, models, applications and future trends. Journal of Management Analytics, 6(1), 1-29. https://doi.org/10.1080/23270012.2019.1570365
  25. Ma, L. (2024). Research on Oral Teaching of International Chinese Language Education Assisted by ChatGPT. Journal of Higher Vocational Education (ISSN: 3005-5784), 1(2), 129.
  26. Ma, R., Qian, M., Tang, S., Bannò, S., Knill, K. M., & Gales, M. J. (2025). Assessment of L2 Oral Proficiency using Speech Large Language Models. arXiv preprint. https://doi.org/10.48550/arXiv.2505.21148
  27. Mahmoud, C. F., & Sørensen, J. T. (2024). Artificial Intelligence in Personalized Learning with a Focus on Current Developments and Future Prospects. Research and Advances in Education, 3(8), 25-31. https://doi.org/10.56397/RAE.2024.08.04
  28. Malfatti, F. I. (2025). Chatgpt, education, and understanding. Social Epistemology, 1-15. https://doi.org/10.1080/02691728.2025.2449599
  29. Mema, B., Basholli, F., & Hyka, D. (2024). Learning transformation and virtual interaction through ChatGPT in Albanian higher education. Advanced Engineering Science, 4, 130-140.
  30. Muthmainnah, M., Asad, M. M., Yakin, A. A., & Almusharraf, N. M. (2025). Human-robot interaction using ChatGPT for technology driven language learning: contextual insights from the higher education institution of Indonesia. Asian Education and Development Studies. https://doi.org/10.1108/AEDS-01-2025-0038
  31. Murtiningsih, S., Sujito, A., & Soe, K. K. (2024). Challenges of using ChatGPT in education: A digital pedagogy analysis. Int J Eval & Res Educ ISSN, 2252(8822), 3467. https://doi.org/10.11591/ijere.v13i5.29467
  32. Najafov, E. (2024). Overview of ChatGPT. In ChatGPT for Marketing: Learn Practical Applications of ChatGPT for Marketing (pp. 1-15). Berkeley, CA: Apress. https://doi.org/10.1007/979-8-8688-0312-3_1
  33. Neupane, K. R. (2025). Strategies of Teaching Speaking Skills in Secondary English Classroom: An Auto/Ethnographic Study [Yüksek lisans tezi, Katmandu Üniversitesi]. Katmandu University Digital Repository.
  34. Owan, V. J., Abang, K. B., Idika, D. O., Etta, E. O., & Bassey, B. A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. Eurasia journal of mathematics, science and technology education, 19(8). https://doi.org/10.29333/ejmste/13428
  35. Page, L. C., & Gehlbach, H. (2017). How an artificially intelligent virtual assistant helps students navigate the road to college. Aera Open. https://doi.org/10.1177/2332858417749220
  36. Pangestu, H. P., & Suwartono, T. (2024). Exploring the Role of Artificial Intelligence (AI) Support in Assisting Students’ English-Speaking Skills. Educalitra: English Education, Linguistics, and Literature Journal, 3(2), 134-147. https://doi.org/10.5281/zenodo.13134979
  37. Park, D. M., & Lee, H. J. (2024). Literature Review of AI Hallucination Research Since the Advent of ChatGPT: Focusing on Papers from arXiv. Informatization Policy, 31(2), 3-38. https://doi.org/10.22693/NIAIP.2024.31.2.003
  38. Salas-Pilco, S. Z., Xiao, K., & Hu, X. (2022). Artificial intelligence and learning analytics in teacher education: A systematic review. Education Sciences, 12(8), 569. https://doi.org/10.3390/educsci12080569
  39. Sayed, B. T., Bani Younes, Z. B., Alkhayyat, A., Adhamova, I., & Teferi, H. (2024). To be with artificial intelligence in oral test or not to be: a probe into the traces of success in speaking skill, psychological well-being, autonomy, and academic buoyancy. Language Testing in Asia, 14(1), 49. https://doi.org/10.1186/s40468-024-00321-0
  40. Schwalbe, N., & Wahl, B. (2020). Artificial intelligence and the future of global health. The Lancet, 395(10236), 1579-1586.
  41. Sreen, A. S., & Majid, M. H. M. (2024). Leveraging ChatGPT for personalized learning: A systematic review in educational settings. Amandemen: Journal of Learning, Teaching and Educational Studies, 2(1), 63-78. https://doi.org/10.61166/amd.v2i1.72
  42. Stahl, B. C., & Eke, D. (2024). The ethics of ChatGPT–Exploring the ethical issues of an emerging technology. International Journal of Information Management, 74, 102700. https://doi.org/10.1016/j.ijinfomgt.2023.102700
  43. Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning & Teaching, 6(1), 1-10. https://doi.org/10.37074/jalt.2023.6.1.17
  44. Surden, H. (2018). Artificial intelligence and law: An overview. Ga. St. UL Rev., 35, 1305.
  45. Tapalova, O., & Zhiyenbayeva, N. (2022). Artificial intelligence in education: AIEd for personalised learning pathways. Electronic Journal of e-Learning, 20(5), 639-653.
  46. Terzopoulos, G., & Satratzemi, M. (2019, September). Voice assistants and artificial intelligence in education. In Proceedings of the 9th Balkan Conference on Informatics (pp. 1-6). https://doi.org/10.1145/3351556.3351588
  47. Vetrivel, S. C., Arun, V. P., Ambikapathi, R., & Saravanan, T. P. (2025). Automated Grading Systems: Enhancing Efficiency and Consistency in Student Assessments. In Adopting Artificial Intelligence Tools in Higher Education (pp. 41-61). CRC Press.
  48. Wang, Y. (2025). A Study on the Efficacy of ChatGPT-4 in Enhancing Students’ English Communication Skills. SAGE Open, 15(1., https://doi.org/10.1177/21582440241310644
  49. Xu, S., Chen, P., & Zhang, G. (2024). Exploring the impact of the use of ChatGPT on foreign language self-efficacy among Chinese students studying abroad: The mediating role of foreign language enjoyment. Heliyon, 10(21). https://doi.org/10.1016/j.heliyon.2024.e39845
  50. Xu, W., Meng, J., Raja, S. K. S., Priya, M. P., & Kiruthiga Devi, M. (2023). Artificial intelligence in constructing personalized and accurate feedback systems for students. International Journal of Modeling, Simulation, and Scientific Computing, 14(01), 2341001. https://doi.org/10.1142/S1793962323410015
  51. Xu, X., Wang, X., Zhang, Y., & Zheng, R. (2024). Applying ChatGPT to tackle the side effects of personal learning environments from learner and learning perspective: An interview of experts in higher education. PLoS One, 19(1). https://doi.org/10.1371/journal.pone.0295646
  52. Yenduri, G., Srivastava, G., Maddikunta, P. K. R., Jhaveri, R. H., Wang, W., Vasilakos, A. V., & Gadekallu, T. R. (2023). Generative pre-trained transformer: A comprehensive review on enabling technologies, potential applications, emerging challenges, and future directions. https://doi.org/10.48550/arXiv.2305.10435
  53. Zaghlool, Z. D., & Khasawneh, M. A. S. (2023). Incorporating the impacts and limitations of AI-driven feedback, evaluation, and real-time conversation tools in foreign language learning. Migration Letters, 20(7).
  54. Zhai, C., Wibowo, S., & Li, L. D. (2024). The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review. Smart Learning Environments, 11(1), 28. https://doi.org/10.1186/s40561-024-00316-7
  55. Zou, B., Liviero, S., Ma, Q., Zhang, W., Du, Y., & Xing, P. (2024). Exploring EFL learners’ perceived promise and limitations of using an artificial intelligence speech evaluation system for speaking practice. System, 126. https://doi.org/10.1016/j.system.2024.103497