SavedAdd your journal

Journals / Turkish Journal of Mathematics and Computer Science / 2025 / Volume: 17 Issue: 2

Fine-Tuning LLaMA2, LLaMA3, and Phi3 Models for Code Generation with Turkish Instructions

Emir Öztürk(TRAKYA ÜNİVERSİTESİ, MÜHENDİSLİK FAKÜLTESİ, BİLGİSAYAR MÜHENDİSLİĞİ BÖLÜMÜ)0000-0002-3734-5171
Aydın Carus(Trakya Üniversitesi, Mühendislik Fakültesi, Bilgisayar Mühendisliği Bölümü)0000-0003-3370-5974
Pages
—
Publication Date
2025-12-30
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The performance of the trained language models has been evaluated using the ROUGE, BLEU and METEOR metrics and models that can generate code are presented

📌 3 Key Takeaways

🎯
Objective & Hypothesis:With the advancement of artificial intelligence and large language models, various models have begun to be utilized at every stage of software development processes, leading to changes in coding habits.
📊
Key Findings & Data:In this study, the “python code instructions 18k alpaca” dataset, which includes data for translation, code generation from scratch, and error querying, has been translated into Turkish, and different language models have been trained on this dataset.
💡
Conclusion & Impact:The performance of the trained language models has been evaluated using the ROUGE, BLEU and METEOR metrics and models that can generate code are presented
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Code generation · Llama · Phi3

Abstract

With the advancement of artificial intelligence and large language models, various models have begun to be utilized at every stage of software development processes, leading to changes in coding habits. Instead of writing entire pieces of code themselves, developers now leave certain patterns to language models or use these models to query issues they encounter. Similarly, code translation from one programming language to another is also carried out with the help of language models. While such studies are observed in the English language, a model designed specifically for generating code in response to queries posed in Turkish does not yet exist. In this study, the “python code instructions 18k alpaca” dataset, which includes data for translation, code generation from scratch, and error querying, has been translated into Turkish, and different language models have been trained on this dataset. The performance of the trained language models has been evaluated using the ROUGE, BLEU and METEOR metrics and models that can generate code are presented

🔖

Cite This Article

Öztürk, E., Carus, A. (2025). Fine-Tuning LLaMA2, LLaMA3, and Phi3 Models for Code Generation with Turkish Instructions. Turkish Journal of Mathematics and Computer Science, Volume: 17 Issue: 2. https://doi.org/10.47000/tjmcs.1699086

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (21)

  1. Abdin, M., Aneja, J., Behl, H., Bubeck, S., Eldan, R. et al., Phi-4 technical report, arXiv preprint arXiv:2412.08905, (2024).
  2. Allamanis, M., Tarlow, D., Gordon, A.,Wei, Y., Bimodal modelling of source code and natural language, International Conference on Machine Learning, PMLR, (2015), 2123–2132.
  3. Allamanis, M., Barr, E.T., Devanbu, P., Sutton, C., A survey of machine learning for big code and naturalness, ACM Computing Surveys, 51(2018), 1–37.
  4. Banerjee, S., Lavie, A., METEOR: An automatic metric for MT evaluation with improved correlation with human judgments, Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, ( 2005), 65–72.
  5. Campbell, B.A., Treude, C., NLP2Code: Code snippet content assist via natural language tasks, Proceedings of the 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME), (2017), 628–632.
  6. Dehaerne, E., Dey, B., Halder, S., De Gendt, S., Meert, W., Code generation using machine learning: A systematic review, IEEE Access, 10(2022), 82434–82455.
  7. Desai, A., Gulwani, S., Hingorani, V., Jain, N., Karkare, A. et al., Program synthesis using natural language, Proceedings of the 38th International Conference on Software Engineering, (2016), 345–356.
  8. Falcini, F., Lami, G., Costanza, A. M., Deep learning in automotive software, IEEE Software, 34(2017), 56–63.
  9. Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X. et al., CodeBERT: A pre-trained model for programming and natural languages, arXiv preprint arXiv:2002.08155, (2020).
  10. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A. et al., The LLaMA 3 herd of models, arXiv preprint arXiv:2407.21783, (2024).
  11. Guo, Q., Xie, X., Li, Y., Zhang, X., Liu, Y. et al., Audee: Automated testing for deep learning frameworks, Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, 2020, 486–498.
  12. Jiang, D., Liu, Y., Liu, S., Zhao, J., Zhang, H. et al., From CLIP to DINO: Visual encoders shout in multi-modal large language models, arXiv preprint arXiv:2310.08825, (2023).
  13. Li, K., Zhu, A., Zhao, P., Song, J., Liu, J., Utilizing deep learning to optimize software development processes, arXiv preprint arXiv:2404.13630, (2024).
  14. Lin, C.-Y., ROUGE: A package for automatic evaluation of summaries, Text Summarization Branches Out, 2004, 74–81.
  15. Narayanan, A., Wang, J., Shi, L., Wei, M., Wang, S., Automatic unit test generation for deep learning frameworks based on API knowledge, arXiv preprint arXiv:2307.00404, (2023).
  16. Papineni, K., Roukos, S., Ward, T., Zhu, W.-J., BLEU: A method for automatic evaluation of machine translation, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, (2002), 311–318.
  17. Royce, W.W., Managing the development of large software systems: concepts and techniques, Proceedings of the 9th International Conference on Software Engineering, (1987), 328–336.
  18. Shin, J., Nam, J., A survey of automatic code generation from natural language, Journal of Information Processing Systems, 17(2021), 537–555.
  19. Sreekanth, N., Rama Devi, J., Shukla, K.A., Mohanty, D.K., Srinivas, A. et al., Evaluation of estimation in software development using deep learning-modified neural network, Applied Nanoscience, 13(2023), 2405–2417.
  20. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A. et al., LLaMA 2: Open foundation and fine-tuned chat models, arXiv preprintarXiv:2307.09288, (2023).
  21. Royce, W.W., Managing the development of large software systems: concepts and techniques, Proceedings of the 9th International Conference on Software Engineering, (1987), 328–336.