Journals / Erzincan Üniversitesi Eğitim Fakültesi Dergisi / 2026 / Cilt: 28 Sayı: 2026

Effects of an AI-Supported LMS on Academic Achievement and Learner Perceptions in Programming Education

Programlama Eğitiminde Yapay Zeka Destekli Bir Öğrenme Yönetim Sisteminin Akademik Başarı ve Öğrenci Algıları Üzerindeki Etkileri

Nihan Arslan Namlı(İSKENDERUN TEKNİK ÜNİVERSİTESİ)0000-0002-5425-1468
Pages
1–11
Publication Date
2026-10-21
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Conflicting findings in the literature regarding this process indicate a need to reconcile divergent evidence in future studies.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:This study investigated the academic success of an AI-supported learning system dedicated to a PHP programming course at the associate degree level and how students experienced this.
📊
Key Findings & Data:Although course-embedded AI feedback is theoretically promising, evidence that jointly addresses academic achievement and learners’ experience in authentic programming courses remains limited. 67 third-semester associate-degree Computer Programming vocational school students, participated in a convergent parallel mixed-methods study over four weeks.
💡
Conclusion & Impact:Conflicting findings in the literature regarding this process indicate a need to reconcile divergent evidence in future studies.
🔬Design:Araştırma Makalesi
📍Scope:Comprehensive Sample
⚙️Variables:Eğitimde yapay zeka · programlama eğitimi · akademik başarı

Abstract

This study investigated the academic success of an AI-supported learning system dedicated to a PHP programming course at the associate degree level and how students experienced this. Although course-embedded AI feedback is theoretically promising, evidence that jointly addresses academic achievement and learners’ experience in authentic programming courses remains limited. 67 third-semester associate-degree Computer Programming vocational school students, participated in a convergent parallel mixed-methods study over four weeks. The experimental group (n =34) worked with the AI-supported LMS, while the control group (n =33) received traditional lecture-based instruction. The quantitative component employed a quasi-experimental design with experimental and control groups. Success was measured with a 20-item multiple-choice test (KR-20=.87) administered before and after the four-week intervention; semi-structured conversations with 12 students were analyzed using content analysis. After controlling for variation in pre-test scores using covariance analysis, experimental group achieved significantly higher post-test scores than those in the control group, F(1, 64) = 5.671, p = .018, ηp² = .081. In the interviews, it was noted that the developments related to instant and error-specific feedback increased their awareness of their errors and helped them to regulate their learning; however, no adaptive content suggestions emerged. The findings demonstrate that feedback quality and progress tracking are essential components of such systems. However, it is of great importance that AI-supported feedback can be used independently by students. Conflicting findings in the literature regarding this process indicate a need to reconcile divergent evidence in future studies.

Özet

Bu çalışma, ön lisans düzeyindeki bir PHP programlama dersine yönelik YZ destekli öğrenme yönetim sisteminin (ÖYS) akademik başarıya etkisini ve öğrencilerin bu süreci nasıl deneyimlediğini incelemiştir. Dört hafta boyunca yürütülen yakınsayan paralel karma yöntem desenli çalışmaya 67 meslek yüksekokulu öğrencisi katılmıştır. Deney grubu (n = 34) YZ destekli ÖYS ile çalışırken kontrol grubu (n = 33) geleneksel anlatıma dayalı öğretim almıştır. Nicel bileşen, deney ve kontrol gruplarını içeren yarı deneysel bir desen olarak kurgulanmıştır. Başarı, uygulama öncesi ve sonrasında uygulanan 20 maddelik bir başarı testiyle (KR-20 = .87) ölçülmüş; 12 öğrenciyle yapılan yarı yapılandırılmış görüşmeler içerik analiziyle çözümlenmiştir. Ön test puanlarındaki değişkenlik kovaryans analizi ile kontrol edildikten sonra deney grubu, kontrol grubuna kıyasla anlamlı düzeyde daha yüksek son test puanları elde etmiştir, F(1, 64) = 5.671, p = .018, ηp² = .081. Görüşmelerde, anlık ve hataya özgü geri bildirimin öğrencilerin hatalarına ilişkin farkındalığını artırdığı ve öğrenmelerini düzenlemelerine yardımcı olduğu; ancak uyarlanabilir içerik önerilerinin ortaya çıkmadığı belirlenmiştir. Bulgular, geri bildirim kalitesi ve ilerleme takibinin bu tür sistemlerin temel bileşenleri olduğunu göstermektedir. Bununla birlikte, YZ destekli geri bildirimin öğrenciler tarafından bağımsız şekilde kullanılabilmesi büyük önem taşımaktadır. Bu sürece ilişkin literatürde yer alan çelişkili bulgular, gelecekte yapılacak araştırmalarda farklı yöndeki kanıtların uzlaştırılması gereksinimine işaret etmektedir.

🔖

Cite This Article

Namlı, N. A. (2026). Effects of an AI-Supported LMS on Academic Achievement and Learner Perceptions in Programming Education. Erzincan Üniversitesi Eğitim Fakültesi Dergisi, Cilt: 28 Sayı: 2026, 1–11. https://doi.org/10.17556/erziefd.1974881

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (40)

  1. Cavalcanti, A. P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y.-S., Gašević, D., & Mello, R. F. (2021). Automatic feedback in online learning environments: A systematic literature review. Computers and Education: Artificial Intelligence, 2, 100027. https://doi.org/10.1016/j.caeai.2021.100027
  2. Chen, A., Wei, Y., Le, H., & Zhang, Y. (2025). Learning by teaching with Chat GPT: The effect of teachable Chat GPT agent on programming education. British Journal of Educational Technology, 57, 163–184. https://doi.org/10.1111/bjet.70001
  3. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203771587
  4. Creswell, J. W., & Plano Clark, V. L. (2022). Designing and conducting mixed methods research (4th ed.). SAGE Publications.
  5. Creswell, J. W., & Poth, C. N. (2018). Qualitative inquiry and research design: Choosing among five approaches (4th ed.). SAGE Publications.
  6. De Boer, H., Donker, A. S., Kostons, D. D., & Van der Werf, G. P. C. (2018). Long-term effects of metacognitive strategy instruction on student academic performance: A meta-analysis. Educational Research Review, 24, 98–115. https://doi.org/10.1016/j.edurev.2018.03.002
  7. Ezzaim, A., Dahbi, A., Haidine, A., & Aqqal, A. (2023). AI-based adaptive learning: A systematic mapping of the literature. Journal of Universal Computer Science, 29(10), 1283–1312. https://doi.org/10.3897/jucs.90528
  8. Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906
  9. Garzón, J., Patiño, E., & Marulanda, C. (2025). Systematic review of artificial intelligence in education: Trends, benefits, and challenges. Multimodal Technologies and Interaction, 9(8), 84. https://doi.org/10.3390/mti9080084
  10. Hao, Q., Smith IV, D. H., Ding, L., Ko, A., Ottaway, C., Wilson, J., Arakawa, K. H., Turcan, A., Poehlman, T., & Greer, T. (2022). Towards understanding the effective design of automated formative feedback for programming assignments. Computer Science Education, 32(1), 105–127. https://doi.org/10.1080/08993408.2020.1860408
  11. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
  12. Heikkinen, S., Saqr, M., Malmberg, J., & Tedre, M. (2023). Supporting self-regulated learning with learning analytics interventions: A systematic literature review. Education and Information Technologies, 28(3), 3059–3088. https://doi.org/10.1007/s10639-022-11281-4
  13. Hooshyar, D., Weng, X., Sillat, P. J., Tammets, K., Wang, M., & Hämäläinen, R. (2024). The effectiveness of personalized technology-enhanced learning in higher education: A meta-analysis with association rule mining. Computers & Education, 223, 105169. https://doi.org/10.1016/j.compedu.2024.105169
  14. Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
  15. Hu, S. (2024). The effect of artificial intelligence-assisted personalized learning on student learning outcomes: A meta-analysis based on 31 empirical research papers. Science Insights Education Frontiers, 24(1), 3873–3894. https://doi.org/10.15354/sief.24.re395
  16. Jing, Y., Wang, H., Chen, X., & Wang, C. (2024). What factors will affect the effectiveness of using Chat GPT to solve programming problems? Humanities and Social Sciences Communications, 11(1), Article 319. https://doi.org/10.1057/s41599-024-02736-9
  17. Keuning, H., Jeuring, J., & Heeren, B. (2018). A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education, 19(1), 1–43. https://doi.org/10.1145/3231711
  18. Kölemen, C. Ş., & Şahin, E. (2026). Yapay zekâ destekli anında geri bildirimin lise öğrencilerinin başarı motivasyonuna etkisi [The effect of AI-supported instant feedback on high school students’ achievement motivation]. Eğitim Teknolojisi Kuram ve Uygulama, 16(1), 131–158.
  19. Lo, C. K., Hew, K. F., & Jong, M. S. Y. (2024). The influence of Chat GPT on student engagement: A systematic review and future research agenda. Computers & Education, 219, Article 105100. https://doi.org/10.1016/j.compedu.2024.105100
  20. Ma, W., Ma, W., Hu, Y., & Bi, X. (2025). The who, why, and how of AI-based chatbots for learning and teaching in higher education: A systematic review. Education and Information Technologies, 30(6), 7781–7805. https://doi.org/10.1007/s10639-024-13128-6
  21. Merriam, S. B. (2002). Qualitative research in practice: Examples for discussion and analysis. Jossey-Bass.
  22. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Sage Publications.
  23. Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090
  24. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). Mc Graw-Hill.
  25. Sezer, C., & Korucu, A. T. (2019). Bilişim teknolojileri ve yazılım dersinde öğrenme yönetim sistemi kullanımının öğrenci akademik başarısına etkisi [The effect of using a learning management system in the information technologies and software course on students’ academic achievement]. Gazi Eğitim Bilimleri Dergisi, 5, 157–176.
  26. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795
  27. Tabachnick, B. G., & Fidell, L. S. (2019). Using multivariate statistics (7th ed.). Pearson.
  28. Theobald, M. (2021). Self-regulated learning training programs enhance university students' academic performance, self-regulated learning strategies, and motivation: A meta-analysis. Contemporary Educational Psychology, 66, 101976. https://doi.org/10.1016/j.cedpsych.2021.101976
  29. Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: Chat GPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15. https://doi.org/10.1186/s40561-023-00237-x
  30. Topham, L., Atherton, P., Reynolds, T., Hussain, Y., Hussain, A., Kolivand, H., & Khan, W. (2025). Artificial intelligence in educational technology: A systematic review of datasets and applications. ACM Computing Surveys. Advance online publication. https://doi.org/10.1145/3768312
  31. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
  32. Wang, L., & Luo, J. (2023). The relationship between student interaction with generative artificial intelligence and learning achievement: Serial mediating roles of self-efficacy and cognitive engagement. Frontiers in Psychology, 14, Article 1285392.https://doi.org/10.3389/fpsyg.2023.1285392
  33. Wisniewski, B., Zierer, K., & Hattie, J. (2019). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 3087. https://doi.org/10.3389/fpsyg.2019.03087
  34. Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x
  35. Xu, W., & Ouyang, F. (2021). A systematic review of AI role in the educational system based on a proposed conceptual framework. Education and Information Technologies, 27, 9455–9478. https://doi.org/10.1007/s10639-021-10774-y
  36. Yıldırım, A., & Şimşek, H. (2021). Sosyal bilimlerde nitel araştırma yöntemleri [Qualitative research methods in the social sciences] (12. baskı). Seçkin Yayıncılık.
  37. Zhang, C. X., Rice, R. E., & Wang, L. H. (2025). College students' literacy, Chat GPT activities, educational outcomes, and trust from a digital divide perspective. New Media & Society.https://doi.org/10.1177/14614448241301741
  38. Zhang, L., Jiang, Q., Xiong, W., & Zhao, W. (2025). Effects of Chat GPT-based human–computer dialogic interaction programming activities on student engagement. Journal of Educational Computing Research, 63(4), 988–1023. https://doi.org/10.1177/07356331251333874
  39. Zheng, L., Long, M., Zhong, L., & Gyasi, J. F. (2022). The effectiveness of technology-facilitated personalized learning on learning achievements and learning perceptions: A meta-analysis. Education and Information Technologies, 27(8), 11807–11830.https://doi.org/10.1007/s10639-022-11092-7
  40. Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. https://doi.org/10.1207/S15430421TIP4102_2