SavedAdd your journal

Journals / Zeki Sistemler Teori ve Uygulamaları Dergisi / 2022 / Cilt: 5 - Sayı: 2

Auxiliary Learning of Non-Monotonic Hyperparameter Scheduling System Via Grid Search

Grid Arama Yoluyla Monotonik Olmayan Hiperparametre Planlama Sisteminin Yardımcı Öğrenimi

Alaa Ali HAMEED
Pages
168–177
Publication Date
2022-09-21
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

A comprehensive set of experiments indicated that the BPVAM is a robust and highly efficient algorithm.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Recent advancements in advanced neural networks have given rise to new adaptive learning strategies.
📊
Key Findings & Data:Conventional learning strategies suffer from many issues, such as slow convergence and lack of robustness.
💡
Conclusion & Impact:A comprehensive set of experiments indicated that the BPVAM is a robust and highly efficient algorithm.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Uyarlanabilir sinir ağları · Hiperparametre · Kararlı durum hatası

Abstract

Recent advancements in advanced neural networks have given rise to new adaptive learning strategies. Conventional learning strategies suffer from many issues, such as slow convergence and lack of robustness. To fully exploit its potential, these issues must be resolved. Both issues are related to the step-size, and momentum term, which is generally fixed and remains uniform for all weights associated with each network layer. In this study, the recently published Back-Propagation Algorithm with Variable Adaptive Momentum (BPVAM) algorithm has been proposed to overcome these issues and improve effectiveness for classification. The study was conducted on various hyperparameters based on the grid search approach, then the optimal values of hyperparameters have trained these algorithms. Six cases were considered with varying values of the hyperparameter to evaluate the impact of the hyperparameter on the training models. It is empirically proven that the convergence behavior of the model is improved in terms of the mean and standard deviation for accuracy and the sum of squared error (SSE). A comprehensive set of experiments indicated that the BPVAM is a robust and highly efficient algorithm.

Özet

Gelişmiş sinir ağlarındaki son gelişmeler, yeni uyarlanabilir öğrenme stratejilerine yol açmıştır. Geleneksel öğrenme stratejileri, yavaş yakınsama ve sağlamlık eksikliği gibi birçok sorundan muzdariptir. Potansiyelinden tam olarak yararlanmak için bu sorunların çözülmesi gerekir. Her iki konu da adım boyutu ve genellikle sabit olan ve her ağ katmanıyla ilişkili tüm ağırlıklar için tek tip kalan momentum terimi ile ilgilidir. Bu çalışmada, bu sorunların üstesinden gelmek ve sınıflandırma etkinliğini artırmak için yakın zamanda yayınlanan Değişken Uyarlanabilir Momentumlu Geri Yayılım Algoritması (BPVAM) algoritması önerilmiştir. Çalışma grid arama yaklaşımına dayalı olarak çeşitli hiperparametreler üzerinde yürütülmüş, daha sonra hiperparametrelerin optimal değerleri bu algoritmaları eğitmiştir. Hiperparametrenin eğitim modelleri üzerindeki etkisini değerlendirmek için hiperparametrenin değişen değerlerine sahip altı durum ele alındı. Modelin yakınsama davranışının, doğruluk için ortalama ve standart sapma ve karesel hatanın toplamı (SSE) açısından iyileştirildiği deneysel olarak kanıtlanmıştır. Kapsamlı bir deney seti, BPVAM'nin sağlam ve yüksek verimli bir algoritma olduğunu gösterdi.

🔖

Cite This Article

HAMEED, A. A. (2022). Auxiliary Learning of Non-Monotonic Hyperparameter Scheduling System Via Grid Search. Zeki Sistemler Teori ve Uygulamaları Dergisi, Cilt: 5 - Sayı: 2, 168–177. https://ajindex.com/articles/882922

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (10)

  1. A. and Newman, D. J. 2007.
  2. UCI Machine Learning Repository, Department of Information and Computer Sciences, University of California, Irvine. Available at www.ics.uci.edu/~mlearn/MLRepository.html. Bengio, Y., Simard, P., Frasconi, P., 1994. Learning Long-Term Dependencies with Gradient Descent is Difficult. IEEE Transactions on Neural Networks 5, 157–166. https://doi.org/10.1109/72.279181
  3. Demircan Keskin, F., Çiçekli, U., İçli, D., 2022. Prediction of Failure Categories in Plastic Extrusion Process with Deep Learning. Journal of Intelligent Systems: Theory and Applications, 5(1), 27–34. https://doi.org/10.38016/jista.878854
  4. Duchi, J., Hazan, E., Singer, Y., 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(7). Erol, B.A., Majumdar, A., Lwowski, J., Benavidez, P., Rad, P., Jamshidi, M., 2018. Improved deep neural network object tracking system for applications in home robotics, in: Studies in Computational Intelligence. Springer Verlag, pp. 369–395. https://doi.org/10.1007/978-3-319-89629-8_14
  5. Gemirter, C. B., Goularas, D., 2021. A Turkish Question Answering System Based on Deep Learning Neural Networks. Journal of Intelligent Systems: Theory and Applications, 4(2), 65–75. https://doi.org/10.38016/jista.815823
  6. Glorot, X., Bengio, Y., 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249-256.
  7. Güney, E., Çakmak, O., Kocaman, Ç., 2022. Classification of Stockwell Transform Based Power Quality Disturbance with Support Vector Machine and Artificial Neural Networks. Journal of Intelligent Systems: Theory and Applications, 5(1), 75–84. https://doi.org/10.38016/jista.996541
  8. Hameed, A.A., Karlik, B., Salman, M.S., 2016. Back-propagation algorithm with variable adaptive momentum. Knowledge-Based Systems 114, 79–87. https://doi.org/10.1016/j.knosys.2016.10.001
  9. He, K., Zhang, X., Ren, S., Sun, J., 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034.
  10. Hertel, L., Collado, J., Sadowski, P., Ott, J., Baldi, P., 2020. Sherpa: Robust hyperparameter optimization for machine learning. SoftwareX, 12, 100591. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. R., 2012. Improving neural networks by preventing co