SavedAdd your journal

Journals / Nevşehir Hacı Bektaş Veli Üniversitesi SBE Dergisi / 2026 / Sayı: YZ (AI) ve Gelecek Perspektifler Özel Sayı

Evaluation of optical character recognition accuracy of Claude Opus 4 in handwritten student compositions

Öğrencilerin el yazısı kompozisyonlarında Claude Opus 4’ün optik karakter tanıma doğruluğunun değerlendirilmesi

Yusuf Emre Yeşilyurt(BURDUR MEHMET AKİF ERSOY ÜNİVERSİTESİ)0000-0002-8520-1359
Pages
142–156
Publication Date
2025-12-07
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The findings revealed high character accuracy (over 98%) and strong word accuracy (over 96%), though sentence coherence was more variable.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:This study evaluates Claude Opus 4 for optical character recognition (OCR) on 30 handwritten English as a Foreign Language (EFL) essay by testing if its fidelity is adequate for AI-supported assessment.
📊
Key Findings & Data:The findings revealed high character accuracy (over 98%) and strong word accuracy (over 96%), though sentence coherence was more variable.
💡
Conclusion & Impact:The study recommends spot-checking outputs with ≥95% word accuracy and fully re-transcribing those below, suggesting that an “AI + Teacher” workflow remains essential for high-stakes use.
🔬Design:Araştırma Makalesi
📍Scope:Comprehensive Sample
⚙️Variables:El Yazısı Tanıma · Otomatik Yazma Değerlendirmesi · Çok Kipli Büyük Dil Modelleri

Abstract

This study evaluates Claude Opus 4 for optical character recognition (OCR) on 30 handwritten English as a Foreign Language (EFL) essay by testing if its fidelity is adequate for AI-supported assessment. For each essay, a human baseline and Claude’s verbatim output were obtained via the web interface. Accuracy was analyzed at character, word, and sentence-coherence levels. The findings revealed high character accuracy (over 98%) and strong word accuracy (over 96%), though sentence coherence was more variable. While word accuracy showed a significant correlation with coherence, character accuracy did not. A 95% word-accuracy threshold was found effective in separating usable from unreliable transcripts. The most common errors were substitutions, punctuation/capitalization, and deletions, while rarer hallucinated insertions (text not present in the original) and semantic distortions disproportionately harmed meaning. The study recommends spot-checking outputs with ≥95% word accuracy and fully re-transcribing those below, suggesting that an “AI + Teacher” workflow remains essential for high-stakes use.

Özet

Bu çalışma, İngilizceyi Yabancı Dil (EFL) olarak öğrenenlerin el yazılarıyla yazdıkları 30 kompozisyon üzerinde Claude Opus 4'ün optik karakter tanıma (OCR) performansını ve bu performansın ortaya koyduğu ürünün yapay zekâ destekli değerlendirmeye uygunluğunu değerlendirmeyi amaçlamıştır. Sınıfta yazılan denemeler fotoğraflanmış ve web arayüzü üzerinden işlenmiştir. Doğruluk; insan referans metinlerine kıyasla karakter, kelime ve cümle tutarlılığı düzeylerinde değerlendirilmiştir. Sonuçlar yüksek karakter (%98 üzeri) ve kelime (%96 üzeri) doğruluğu göstermiş, ancak cümle tutarlılığı değişkenlik göstermiştir. Kelime doğruluğu tutarlılık ile anlamlı bir ilişki gösterirken, karakter doğruluğu göstermemiştir. %95'lik bir kelime doğruluğu eşiğinin, kullanılabilir transkriptleri ayırt etmede etkili olduğu bulunmuştur. Yaygın hatalar arasında yer değiştirme, noktalama/büyük harf ve silme yer alırken; daha nadir görülen halüsinasyonlar ve anlamsal bozulmalar anlamı orantısız şekilde etkilemiştir. Çalışma, "Yapay Zekâ (YZ) + Öğretmen" iş akışının gerekliliğini koruduğunu öne sürmekte, %95 kelime doğruluğunun üzerindeki çıktıların nokta atışı kontrol edilmesini ve bu oranın altındakilerin yeniden transkribe edilmesini önermektedir

🔖

Cite This Article

Yeşilyurt, Y. E. (2025). Evaluation of optical character recognition accuracy of Claude Opus 4 in handwritten student compositions. Nevşehir Hacı Bektaş Veli Üniversitesi SBE Dergisi, Sayı: YZ (AI) ve Gelecek Perspektifler Özel Sayı, 142–156. https://doi.org/10.30783/nevsosbilen.1770463

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (25)

  1. Aljishi, F., Mughaus, R., Luqman, H., & Parvez, M. T. (2024). A comparative study of four handwritten text recognition models in Arabic script. Ingénierie des Systèmes d’Information, 29(6), 2243–2250. https://doi.org/10.18280/isi.290614
  2. AlKendi, W., Gechter, F., Heyberger, L., & Guyeux, C. (2024). Advancements and challenges in handwritten text recognition: A comprehensive survey. Journal of Imaging, 10(1), 18. https://doi.org/10.3390/jimaging10010018
  3. Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745
  4. Biber, D., Nekrasova, T., & Horn, B. (2011). The effectiveness of feedback for L1‐English and L2‐writing development: A meta‐analysis. ETS Research Report Series, 2011(1), i-99. https://doi.org/10.1002/j.2333-8504.2011.tb02241.x
  5. Chan, C. K. Y. (2023). A systematic review – handwritten examinations are becoming outdated, is it time to change to typed examinations in our assessment policy? Assessment & Evaluation in Higher Education, 48(8), 1385–1401. https://doi.org/10.1080/02602938.2023.2219422
  6. Council of Europe. (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment—Companion volume. Council of Europe Publishing
  7. Guo, K., Wang, D. To resist it or to embrace it? Examining ChatGPT’s potential to support teacher feedback in EFL writing. Education and Information Technologies 29, 8435–8463 (2024). https://doi.org/10.1007/s10639-023-12146-0
  8. Hamdi, A., Linhares Pontes, E., Sidere, N., Coustaty, M., & Doucet, A. (2023). In-depth analysis of the impact of OCR errors on named entity recognition and linking. Natural Language Engineering, 29(2), 425–448. https://doi.org/10.1017/S1351324922000110
  9. Han, J., & Li, M. (2024). Exploring ChatGPT-supported teacher feedback in the EFL context. System, 126, 103502. https://doi.org/10.1016/j.system.2024.103502
  10. Han, Y., & Hyland, F. (2015). Exploring learner engagement with written corrective feedback in a Chinese tertiary EFL classroom. Journal of Second Language Writing, 30, 31–44. https://doi.org/10.1016/j.jslw.2015.08.002
  11. Han, T., & Sarı, E. (2022). An investigation on the use of automated feedback in Turkish EFL students’ writing classes. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2067179
  12. Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3), 90–95. https://doi.org/10.1109/MCSE.2007.55
  13. Karatay, Y., & Karatay, L. (2024). Automated writing evaluation use in second language classrooms: A research synthesis. System, 123, 103332. https://doi.org/10.1016/j.system.2024.103332
  14. Kellogg, R. T., Whiteford, A. P., & Quinlan, T. (2010). Does Automated Feedback Help Students Learn to Write? Journal of Educational Computing Research, 42(2), 173-196. https://doi.org/10.2190/EC.42.2.c
  15. Kim, S., Baudru, J., Ryckbosch, W., Bersini, H., & Ginis, V. (2025). Early evidence of how LLMs outperform traditional systems on OCR/HTR tasks for historical records. arXiv preprint. https://arxiv.org/abs/2501.11623
  16. McKinney, W. (2010). Data structures for statistical computing in Python. In S. van der Walt & J. Millman (Eds.), Proceedings of the 9th Python in Science Conference (SciPy 2010) (pp. 56–61)
  17. Michail, A., Opitz, J., Wang, Y., Meister, R., Sennrich, R., & Clematide, S. (2025). Cheap character noise for OCR-robust multilingual embeddings. Findings of the Association for Computational Linguistics: ACL 2025, 11705–11716
  18. Ramani, K., Maheswari, G. U., Krishna, K. P., Meghashyam, S. V., Komirisetty, V. P. K., & Duraiswamy, Y. (2024). Automatic grading of answer sheets using machine learning techniques. In K. R. Madhavi, P. Subba Rao, J. Avanija, I. L. Manikyamba, & B. Unhelkar (Eds.), Proceedings of the International Conference on Computational Innovations and Emerging Trends (ICCIET 2024) (pp. 275–284). Atlantis Press. https://doi.org/10.2991/978-94-6463-471-6_27
  19. Schaefer, E. (2008). Rater bias patterns in an EFL writing assessment. Language Testing, 25(4), 465-493. https://doi.org/10.1177/0265532208094273
  20. Shen, R., & Chong, S. W. (2023). Learner engagement with written corrective feedback in ESL and EFL contexts: A qualitative research synthesis using a perception-based framework. Assessment & Evaluation in Higher Education, 48(3), 276–290. https://doi.org/10.1080/02602938.2022.2072468
  21. Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students’ writing. Learning and Instruction, 91, 101894. https://doi.org/10.1016/j.learninstruc.2024.101894
  22. Van Rossum, G., & Drake, F. L., Jr. (2009). Python 3 reference manual. CreateSpace
  23. Wilson, J., & Czik, A. (2016). Automated essay evaluation software in English Language Arts classrooms: Effects on teacher feedback, student motivation, and writing quality. Computers & Education, 100, 94–109. https://doi.org/10.1016/j.compedu.2016.05.004
  24. Yeşilyurt, Y. E., & Sevli, O. (2024). Evaluating the accuracy of optical character recognition in ChatGPT for handwritten student essays. In İ. Erpay (Ed.), Proceedings of the 2nd BİLSEL International Kibyra Scientific Researches Congress (pp. 195–201). Astana Publications.
  25. Yu, R., & Yang, L. (2021). ESL/EFL learners’ responses to teacher written feedback: Reviewing a recent decade of empirical studies. Frontiers in Psychology, 12, 735101. https://doi.org/10.3389/fpsyg.2021.735101