SavedAdd your journal

Journals / Akıllı Ulaşım Sistemleri ve Uygulamaları Dergisi / 2026 / Cilt: 9 Sayı: 1

Reliable Information Access in Intelligent Transportation Systems and Autonomous Driving Protocols: An Experimental Evaluation of the RAG Architecture's Regulatory Analysis

Akıllı Ulaşım Sistemleri ve Otonom Sürüş Protokollerinde Güvenilir Bilgi Erişimi: RAG Mimarisinin Mevzuat Analizi Üzerine Deneysel Bir Değerlendirme

Ahmet Akkaya(BANDIRMA ONYEDİ EYLÜL ÜNİVERSİTESİ)0000-0003-4836-2310
Pages
22–41
Publication Date
2026-03-25
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The findings reveal that the RAG architecture provided a 16.65% improvement in semantic similarity scores compared to standard models.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The rapid advancement in Intelligent Transportation Systems (ITS) and autonomous driving technologies necessitates the accurate analysis of complex technical protocols and legal regulations.
📊
Key Findings & Data:While Large Language Models (LLMs) offer significant potential for processing these texts, the risk of "hallucinations" remains a critical barrier in safety-critical domains such as autonomous vehicles.
💡
Conclusion & Impact:The findings reveal that the RAG architecture provided a 16.65% improvement in semantic similarity scores compared to standard models.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Geri Getirme Destekli Nesil (RAG) · Akıllı Ulaşım Sistemleri · Otonom Sürüş Mevzuatı

Abstract

The rapid advancement in Intelligent Transportation Systems (ITS) and autonomous driving technologies necessitates the accurate analysis of complex technical protocols and legal regulations. While Large Language Models (LLMs) offer significant potential for processing these texts, the risk of "hallucinations" remains a critical barrier in safety-critical domains such as autonomous vehicles. This study aims to experimentally evaluate the performance of Retrieval-Augmented Generation (RAG) architectures on key ITS documents, including the Turkish Highway Traffic Law, Euro NCAP protocols, and National ITS Strategy papers. Within the scope of the research, standard LLMs and the proposed RAG system were compared across 44 unique scenarios derived from five distinct document types. The findings reveal that the RAG architecture provided a 16.65% improvement in semantic similarity scores compared to standard models. Statistical analyses confirmed that this performance increase is highly significant (p = 0.0072) with an effect size of Cohen’s d = 0.30. The results demonstrate that RAG systems substantially increase information integrity, offering a reliable solution for regulatory compliance and decision-support mechanisms in the autonomous driving ecosystem. By providing a localised legislation-oriented benchmark, this study fills a significant gap in the literature regarding safety-critical information retrieval.

Özet

Akıllı Ulaşım Sistemleri (AUS) ve otonom sürüş teknolojilerindeki hızlı gelişim, karmaşık teknik protokollerin ve hukuki mevzuatların doğru analiz edilmesini zorunlu kılmaktadır. Büyük Dil Modelleri (LLM), bu metinlerin işlenmesinde büyük potansiyel sunsa da "halüsinasyon" riski otonom araçlar gibi güvenlik-kritik alanlarda ciddi bir engel teşkil etmektedir. Bu çalışmanın amacı, Geri Getirme Destekli Nesil (RAG) mimarisinin; Karayolları Trafik Kanunu, Euro NCAP protokolleri ve Ulusal AUS Strateji belgeleri üzerindeki performansını deneysel olarak değerlendirmektir. Çalışma kapsamında, 5 farklı doküman tipinden türetilen 44 özgün senaryo üzerinden standart LLM ve önerilen RAG sistemi karşılaştırılmıştır. Elde edilen bulgular, RAG mimarisinin anlamsal benzerlik skorlarında standart modellere oranla %16,65 düzeyinde bir iyileşme sağladığını ortaya koymuştur. Yapılan istatistiksel analizler, bu performans artışının p = 0,0072 değeri ile yüksek düzeyde anlamlı olduğunu ve Cohen’s d = 0,30 etki büyüklüğüne sahip olduğunu kanıtlamıştır. Sonuçlar, RAG sistemlerinin bilgi doğruluğunu artırarak otonom sürüş ekosisteminde mevzuat uyumu ve karar destek mekanizmaları için güvenilir bir çözüm sunduğunu göstermektedir. Bu çalışma, yerel mevzuat odaklı bir benchmark sunarak literatürdeki önemli bir boşluğu doldurmaktadır.

🔖

Cite This Article

Akkaya, A. (2026). Reliable Information Access in Intelligent Transportation Systems and Autonomous Driving Protocols: An Experimental Evaluation of the RAG Architecture's Regulatory Analysis. Akıllı Ulaşım Sistemleri ve Uygulamaları Dergisi, Cilt: 9 Sayı: 1, 22–41. https://doi.org/10.51513/jitsa.1848120

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (53)

  1. Akkaya, A., & Közkurt, C. (2025). An effective approach for adaptive operator selection and comparison for PSO algorithm. Cluster Computing, 28(6), 368.
  2. Alshemali, B., & Kalita, J. (2020). Improving the Reliability of Deep Neural Networks in NLP: A Review. Knowledge-Based Systems, 191, 105210. https://doi.org/10.1016/j.knosys.2019.105210
  3. Andriopoulos, K., & Pouwelse, J. (2023). Augmenting LLMs with Knowledge: A survey on hallucination prevention (No. arXiv:2309.16459). arXiv. https://doi.org/10.48550/arXiv.2309.16459
  4. Awadid, A., Becquart, M., Gagnant, M., & Meyer-Vitali, A. (2025). A Retrieval-Augmented Generation (RAG) System for Supporting Architectural Design of Intelligent Transportation Systems. In 2025 25th International Conference on Software Quality, Reliability, and Security Companion (QRS-C) (pp. 1-10). IEEE.
  5. Backgård, W. (2025). The Calculated Law Exploring the limits of transformer models in legal reasoning. https://gupea.ub.gu.se/handle/2077/87586
  6. Bagloee, S. A., Tavana, M., Asadi, M., & Oliver, T. (2016). Autonomous vehicles: Challenges, opportunities, and future implications for transportation policies. Journal of Modern Transportation, 24(4), 284–303. https://doi.org/10.1007/s40534-016-0117-3
  7. Balu, B. V., Geissler, F., Carella, F., Zacchi, J. V., Jiru, J., Mata, N., & Stolle, R. (2025, May). Towards automated safety requirements derivation using agent-based rag. In Proceedings of the AAAI Symposium Series (Vol. 5, No. 1, pp. 299-307).
  8. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. https://doi.org/10.1145/3442188.3445922
  9. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., … Amodei, D. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
  10. Cui, J., Ning, M., Li, Z., Chen, B., Yan, Y., Li, H., Ling, B., Tian, Y., & Yuan, L. (2023). ChatLaw: Open-source legal large language model with integrated external knowledge bases. arXiv. https://doi.org/10.48550/arXiv.2306.16092
  11. Hong, G., Ouyang, T., Zhao, K., Zhou, Z., & Chen, X. (2025). CoEdge-RAG: Optimizing Hierarchical Scheduling for Retrieval-Augmented LLMs in Collaborative Edge Computing. In 2025 IEEE Real-Time Systems Symposium (RTSS) (pp. 162-174). IEEE.
  12. Dilek, E., Talih, Ö., & Ceylan, H. (2023). Ulusal Akıllı Ulaşım Sistemleri Mimarisinin Yaygınlaştırılması: Türkiye Önerisi. Akıllı Ulaşım Sistemleri ve Uygulamaları Dergisi, 6(2), 353–392. https://doi.org/10.51513/jitsa.1309583
  13. Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., & Larson, J. (2025). From Local to Global: A Graph RAG Approach to Query-Focused Summarization (No. arXiv:2404.16130). arXiv. https://doi.org/10.48550/arXiv.2404.16130
  14. Filipovska, E., Mladenovska, A., Dobreva, J., Kitanovski, D., Mitrov, G., Lameski, P., & Zdravevski, E. (2025). Evaluation of Vector Databases and LLMs in RAG-Based Multi-document Question Answering. In B. Risteska Stojkoska & S. Janeska Sarkanjac (Eds), ICT Innovations 2024. TechConvergence: AI, Business, and Startup Synergy (pp. 3–18). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-86162-8_1
  15. Finardi, P., Avila, L., Castaldoni, R., Gengo, P., Larcher, C., Piau, M., Costa, P., & Caridá, V. (2024). The Chronicles of RAG: The Retriever, the Chunk and the Generator (No. arXiv:2401.07883). arXiv. https://doi.org/10.48550/arXiv.2401.07883
  16. Fraade-Blanar, L., Blumenthal, M. S., Anderson, J. M., & Kalra, N. (2018). Measuring Automated Vehicle Safety: Forging a Framework. https://www.rand.org/pubs/research_reports/RR2662.html
  17. Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2024). Retrieval-Augmented Generation for Large Language Models: A Survey (No. arXiv:2312.10997). arXiv. https://doi.org/10.48550/arXiv.2312.10997
  18. Guha, N., Nyarko, J., Ho, D., Ré, C., Chilton, A., K, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D., Zambrano, D., Talisman, D., Hoque, E., Surani, F., Fagan, F., Sarfaty, G., Dickinson, G., Porat, H., Hegland, J., … Li, Z. (2023). LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 36, 44123–44279.
  19. Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T. (2025). A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Trans. Inf. Syst., 43(2), 42:1-42:55. https://doi.org/10.1145/3703155
  20. Huly, O., Pogrebinsky, I., Carmel, D., Kurland, O., & Maarek, Y. (2024). Old IR Methods Meet RAG. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2559–2563. https://doi.org/10.1145/3626772.3657935
  21. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of Hallucination in Natural Language Generation. ACM Comput. Surv., 55(12), 248:1-248:38. https://doi.org/10.1145/3571730
  22. Jiang, Z., Ma, X., & Chen, W. (2024). LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs (No. arXiv:2406.15319). arXiv. https://doi.org/10.48550/arXiv.2406.15319
  23. Jin, B., Yoon, J., Han, J., & Arik, S. O. (2024). Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (No. arXiv:2410.05983). arXiv. https://doi.org/10.48550/arXiv.2410.05983
  24. Juvekar, K., & Purwar, A. (2024). COS-Mix: Cosine Similarity and Distance Fusion for Improved Information Retrieval (No. arXiv:2406.00638). arXiv. https://doi.org/10.48550/arXiv.2406.00638
  25. Kalra, N., & Paddock, S. M. (2016). Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice, 94, 182–193. https://doi.org/10.1016/j.tra.2016.09.010
  26. Kang, L. (2024). Exploring a data-driven framework for safety performance management: A theoretical investigation at the enterprise level. Journal of Loss Prevention in the Process Industries, 91, 105384. https://doi.org/10.1016/j.jlp.2024.105384
  27. Karacan, H. ve Akçay, M. (2019). Mevzuat metinlerinin anlamsal aranması için ontoloji tabanlı bir yaklaşım. Bilişim Teknolojileri Dergisi, 12(4), 325-334. https://doi.org/10.17671/gazibtd.553258
  28. Katz, D. M., Bommarito, M. J., Gao, S., & Arredondo, P. (2024). GPT-4 passes the bar exam. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 382(2270), 20230254. https://doi.org/10.1098/rsta.2023.0254
  29. Koopman, P., & Wagner, M. (2016). Challenges in Autonomous Vehicle Testing and Validation. SAE International Journal of Transportation Safety, 4(1), 15–24.
  30. Korkusuz, M. E., & Karamete, A. (2017). MMORPG Türünde Geliştirilen Bir Eğitsel Oyunun Basit Elektrik Devreleri Ünitesine Uygulanması ve Çeşitli Değişkenler Bakımından İncelenmesi. Eskişehir Osmangazi Üniversitesi Türk Dünyası Uygulama ve Araştırma Merkezi Eğitim Dergisi, 2(1), 78–96.
  31. Lee, J., Chen, A., Dai, Z., Dua, D., Sachan, D. S., Boratko, M., Luan, Y., Arnold, S. M. R., Perot, V., Dalmia, S., Hu, H., Lin, X., Pasupat, P., Amini, A., Cole, J. R., Riedel, S., Naim, I., Chang, M.-W., & Guu, K. (2024). Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? (No. arXiv:2406.13121). arXiv. https://doi.org/10.48550/arXiv.2406.13121
  32. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
  33. Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638
  34. Malik, S., Khan, M. A., El-Sayed, H., Khan, J., & Ullah, O. (2023). How Do Autonomous Vehicles Decide? Sensors, 23(1), 317. https://doi.org/10.3390/s23010317
  35. Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., & Hajishirzi, H. (2023). When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. In A. Rogers, J. Boyd-Graber, & N. Okazaki (Eds), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 9802–9822). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.acl-long.546
  36. Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2021). Generation-augmented retrieval for open-domain question answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 4089-4100).
  37. Marín, J. (2025). Semantic Grounding Index: Geometric Bounds on Context Engagement in RAG Systems (No. arXiv:2512.13771). arXiv. https://doi.org/10.48550/arXiv.2512.13771
  38. Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., & Gao, J. (2025). Large Language Models: A Survey (No. arXiv:2402.06196). arXiv. https://doi.org/10.48550/arXiv.2402.06196
  39. Mozaffari, S., Al-Jarrah, O. Y., Dianati, M., Jennings, P., & Mouzakitis, A. (2022). Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review. IEEE Transactions on Intelligent Transportation Systems, 23(1), 33–47. https://doi.org/10.1109/TITS.2020.3012034
  40. Nicastro, R., Sardu, A., Panchaud, N., & De Virgilio, C. (2017). The Architecture of the Rag GTPase Signaling Network. Biomolecules, 7(3), 48. https://doi.org/10.3390/biom7030048
  41. Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., & Shoham, Y. (2023). In-Context Retrieval-Augmented Language Models. Transactions of the Association for Computational Linguistics, 11, 1316–1331. https://doi.org/10.1162/tacl_a_00605
  42. Rau, D., Wang, S., Déjean, H., & Clinchant, S. (2024). Context Embeddings for Efficient Answer Generation in RAG (No. arXiv:2407.09252). arXiv. https://doi.org/10.48550/arXiv.2407.09252 Rawte, V., Sheth, A., & Das, A. (2023). A Survey of Hallucination in Large Foundation Models (No. arXiv:2309.05922). arXiv. https://doi.org/10.48550/arXiv.2309.05922
  43. Reimers, N., & Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 3982-3992).
  44. Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., & Yih, W. (2023). REPLUG: Retrieval-Augmented Black-Box Language Models (No. arXiv:2301.12652). arXiv. https://doi.org/10.48550/arXiv.2301.12652
  45. Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021). Retrieval Augmentation Reduces Hallucination in Conversation (No. arXiv:2104.07567). arXiv. https://doi.org/10.48550/arXiv.2104.07567
  46. Singh, J. (2023). The Ethical Implications of AI and RAG Models in Content Generation: Bias, Misinformation, and Privacy Concerns. Journal of Science & Technology, 4(1), 156–170.
  47. van Ratingen, M. R. (2017). The Euro NCAP Safety Rating. In A. Piskun (Ed.), Karosseriebautage Hamburg 2017 (pp. 11–20). Springer Fachmedien. https://doi.org/10.1007/978-3-658-18107-9_2
  48. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł. ukasz, & Polosukhin, I. (2017). Attention is All you Need. Advances in Neural Information Processing Systems, 30. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
  49. Wu, D., Nie, L., Mumtaz, R. A., & Agarwal, K. (2025). A LLM-Based Hybrid-Transformer Diagnosis System in Healthcare. IEEE Journal of Biomedical and Health Informatics, 29(9), 6428–6439. https://doi.org/10.1109/JBHI.2024.3481412
  50. Yan, S.-Q., Gu, J.-C., Zhu, Y., & Ling, Z.-H. (2024). Corrective Retrieval Augmented Generation (No. arXiv:2401.15884). arXiv. https://doi.org/10.48550/arXiv.2401.15884
  51. Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., Boyd, J., Boyd, K., Hirsch, K., Langlotz, C., Lee, R., Melia, J., Nelson, J., Sallam, K., Tullis, S., … Hiesinger, W. (2024). Almanac—Retrieval-Augmented Language Models for Clinical Medicine. NEJM AI, 1(2), AIoa2300068. https://doi.org/10.1056/AIoa2300068
  52. Zhao, B., Jin, W., Ser, J. D., & Yang, G. (2023). ChatAgri: Exploring Potentials of ChatGPT on Cross-linguistic Agricultural Text Classification (No. arXiv:2305.15024). arXiv. https://doi.org/10.48550/arXiv.2305.15024
  53. Zhong, Z., Liu, H., Cui, X., Zhang, X., & Qin, Z. (2025). Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation. In O. Rambow, L. Wanner, M. Apidianaki, H. Al-Khalifa, B. D. Eugenio, & S. Schockaert (Eds), Proceedings of the 31st International Conference on Computational Linguistics (pp. 5756–5774). Association for Computational Linguistics. https://aclanthology.org/2025.coling-main.384/