Dergiler / Artificial Intelligence Theory and Applications / 2026 / Volume: 5 Issue: 1

AI-Enhanced Test Automation Tool for Desktop Applications

Sayfa
—
Yayın Tarihi
2025-05-01
DOI
—
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

This approach demonstrates the power of artificial intelligence and data analytics in test automation processes, serving as a significant example for both educational and practical applications.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:Test automation is an essential part of the software testing process.
📊
Temel Bulgular & Veri:The detection of each object in the graphical user interfaces of the software will be carried out using the object detection capabilities of YOLOv9 and Faster R-CNN models.
💡
Sonuç & Katkı:This approach demonstrates the power of artificial intelligence and data analytics in test automation processes, serving as a significant example for both educational and practical applications.
🔬Tür:Modelleme & Sayısal Simülasyon
📍Kapsam:Genişletilmiş Akademik Örneklem
⚙️Değişkenler:Object Detection · Data Analysis · Test Automation

Abstract

Test automation is an essential part of the software testing process. This study aims to develop an AI-enhanced test automation tool for testing the user interfaces of desktop applications. The detection of each object in the graphical user interfaces of the software will be carried out using the object detection capabilities of YOLOv9 and Faster R-CNN models. The study emphasizes the importance of preprocessing steps for achieving successful outcomes in object detection processes. These preprocessing steps include image resizing, data augmentation techniques, and balancing the dataset. Additionally, the correct selection and optimization of hyperparameters (e.g., learning rate, number of epochs, network depth, and anchor box dimensions) in object detection models play a critical role in improving model performance. In this study, data analysis techniques using Python were utilized for hyperparameter optimization. Hyperparameters were evaluated and optimized based on metrics such as model accuracy, loss curves, and training time. As a result, high performance was achieved for both the test automation tool and the object detection process. This approach demonstrates the power of artificial intelligence and data analytics in test automation processes, serving as a significant example for both educational and practical applications.

🔖

Bu Makaleye Atıf Yap

Cömertler, N. C. (2025). AI-Enhanced Test Automation Tool for Desktop Applications. Artificial Intelligence Theory and Applications, Volume: 5 Issue: 1. https://ajindex.com/articles/1609598

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (15)

  1. J. Chen, M. Xie, Z. Xing, C. Chen, X. Xu, L. Zhu, and G. Li, "Object detection for graphical user interface: old fashioned or deep learning or a combination" in Proc. of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020), New York, NY, USA, 2020, pp. 1202-1214. [Online]. https://doi.org/10.1145/3368089.3409691
  2. Dwivedi, S. K., & Rawat, B. (2015, October). A review paper on data preprocessing: A critical phase in web usage mining process. In 2015 International Conference on Green Computing and Internet of Things (ICGCIoT) (pp. 506-510). IEEE.
  3. C. Zhang, T. Shi, J. Ai, and W. Tian, "Construction of GUI Elements Recognition Model for AI Testing based on Deep Learning," in Proc. 2021 8th International Conference on Dependable Systems and Their Applications (DSA), Yinchuan, China, 2021, pp. 508-515. doi: 10.1109/DSA52907.2021.00075.
  4. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), 1, I-511-I-518.
  5. Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), 1, 886-893.
  6. Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010). Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 32(9), 1627-1645
  7. Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 39(6), 1137–1149.
  8. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision (ECCV 2016), 21–37
  9. Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934
  10. Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014), 580–587.
  11. Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(60), 1–48.
  12. Little, R. J., & Rubin, D. B. (2019). Statistical analysis with missing data (Vol. 793). John Wiley & Sons.
  13. Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13, 281–305.
  14. G. Li, G. Baechler, M. Tragut, and Y. Li, "Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale," in Proc. of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22), New York, NY, USA, 2022, Article 67, pp. 1-13. [Online]. https://doi.org/10.1145/3491102.3502042
  15. Huang, Jonathan et al. “Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors.” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016): 3296-3297.