SavedAdd your journal

Journals / International Journal of 3D Printing Technologies and Digital Industry / 2026 / Cilt: 10 Sayı: 2

AN EMPIRICAL ANALYSIS OF HYPERPARAMETER INTERACTIONS IN LIGHTWEIGHT CONVOLUTIONAL NEURAL NETWORKS: OPTIMIZATION, SCHEDULER, AND INITIALIZATION MATRIX

HAFİF EVRİŞİMLİ SİNİR AĞLARINDA HİPERPARAMETRE ETKİLEŞİMLERİNİN AMPİRİK ANALİZİ: OPTİMİZASYON, ZAMANLAYICI VE İLKLENDİRME MATRİSİ

Hüseyin Eldem(KARAMANOĞLU MEHMETBEY ÜNİVERSİTESİ)0000-0002-7333-8104
Pages
455–465
Publication Date
2026-08-31
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

These complex parametric interactions were rigorously verified using a 3-Way ANOVA test and extensive interaction profiles (p < 0,001), providing a definitive empirical guideline for lightweight architectures.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The success of deep learning models depends directly on the selection of core hyperparameters, including weight initialization methods, optimization algorithms, and learning rate schedulers.
📊
Key Findings & Data:In this study, a comprehensive grid search consisting of 240 independent training runs was conducted on the canonical CIFAR-10 dataset using an isolated, shallow Convolutional Neural Network (CNN) architecture stripped of regularizer layers (Batch Normalization, Dropout, etc.).
💡
Conclusion & Impact:These complex parametric interactions were rigorously verified using a 3-Way ANOVA test and extensive interaction profiles (p < 0,001), providing a definitive empirical guideline for lightweight architectures.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Hiperparametre Etkileşimi · Evrişimli Sinir Ağları · OneCycleLR Uyumsuzluğu

Abstract

The success of deep learning models depends directly on the selection of core hyperparameters, including weight initialization methods, optimization algorithms, and learning rate schedulers. In this study, a comprehensive grid search consisting of 240 independent training runs was conducted on the canonical CIFAR-10 dataset using an isolated, shallow Convolutional Neural Network (CNN) architecture stripped of regularizer layers (Batch Normalization, Dropout, etc.). Empirical findings demonstrate that the combinatorial interactions of hyperparameters, rather than their individual effects, dominate model performance. According to the experimental results, the combination of Xavier Normal initialization, AdamW optimizer, and CosineAnnealingLR scheduler exhibited the highest test accuracy at 73.84%. As the prominent finding of this study, termed the 'OneCycleLR local parametric mismatch', it was discovered that the OneCycleLR scheduler, which is frequently recommended in the literature to accelerate convergence, stalls the weight update process of the model when paired with the RMSprop optimizer (dropping to a 10.00% random guess level); conversely, it rescues Pure_SGD and SGD_Momentum variants from sub-optimal plateaus, boosting their performance to 68.97%. These complex parametric interactions were rigorously verified using a 3-Way ANOVA test and extensive interaction profiles (p < 0,001), providing a definitive empirical guideline for lightweight architectures.

Özet

Derin öğrenme modellerinin başarısı; ağırlık ilklendirme yöntemleri, optimizasyon algoritmaları ve öğrenme oranı zamanlayıcıları gibi temel hiperparametrelerin doğru seçilmesine doğrudan bağlıdır. Bu çalışmada, düzenlileştirici katmanlardan (Batch Normalization, Dropout vb.) arındırılmış izole ve hafif/sığ bir Evrişimli Sinir Ağı (CNN) mimarisi üzerinde, kanonik CIFAR-10 veri kümesi kullanılarak 240 bağımsız eğitim koşusundan oluşan kapsamlı bir ızgara araması yürütülmüştür. Ampirik bulgular, hiperparametrelerin tekil etkilerinden ziyade kombinatoryal etkileşimlerinin model başarımı üzerinde domine edici olduğunu göstermiştir. Deneysel sonuçlara göre; Xavier Normal ilklendirme, AdamW optimizörü ve CosineAnnealingLR zamanlayıcı kombinasyonu %73,84 ile en yüksek test doğruluğunu sergilemiştir. Çalışmanın öne çıkan bulgusu olan, OneCycleLR lokal parametrik uyumsuzluğu uyarınca; literatürde yakınsamayı hızlandırdığı için sıklıkla önerilen OneCycleLR zamanlayıcısının RMSprop optimizörü ile eşleştiğinde modelin ağırlık güncelleme sürecini durdurduğunu (%10,00 rastgele tahmin düzeyi), buna karşın Pure_SGD ve SGD_Momentum varyantlarını alt-optimal platolardan kurtararak başarımlarını %68,97' ye çıkardığı keşfedilmiştir. Gözlemlenen bu karmaşık parametrik etkileşimler 3-Yönlü ANOVA testi ve detaylı profil matrisleri ile istatistiksel olarak doğrulanmış (p < 0,001) ve sığ mimarilerle çalışan araştırmacılar için ampirik bir seçim kılavuzu sunulmuştur.

🔖

Cite This Article

Eldem, H. (2026). AN EMPIRICAL ANALYSIS OF HYPERPARAMETER INTERACTIONS IN LIGHTWEIGHT CONVOLUTIONAL NEURAL NETWORKS: OPTIMIZATION, SCHEDULER, AND INITIALIZATION MATRIX. International Journal of 3D Printing Technologies and Digital Industry, Cilt: 10 Sayı: 2, 455–465. https://doi.org/10.46519/ij3dptdi.1962544

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (16)

  1. Zaheer, R., Shaziya, H., "GPU-based empirical evaluation of activation functions in convolutional neural networks", Proceedings of the 2nd International Conference on Inventive Systems and Control (ICISC 2018), Pages 769-773, Coimbatore, 2018.
  2. Delibaş, E., "Rethinking differentiability: a comparative study of smooth vs non-smooth activation functions in CNNs", Duzce University Journal of Science and Technology, Vol. 14, Issue 1, Pages 189-198, 2026.
  3. Zhang, W., Zhang, Y., Zheng, Y., Mo, W., "A general framework for activation function optimization based on mollification theory", Mathematics, Vol. 14, Issue 1, Page 72, 2025.
  4. Isong, N., "Building efficient lightweight CNN models", arXiv:2501.15547, https://arxiv.org/abs/2501.15547v1, Accessed: May 25, 2026, 2025.
  5. Wang, Z., Sun, D., Gong, K., Wang, W., Sun, P., "A lightweight CNN architecture for automatic modulation classification", Electronics, Vol. 10, Issue 21, Page 2679, 2021.
  6. Qian, C., Ling, T., Cichiwskyj, C., Schiele, G., "Configuration-aware approaches for enhancing energy efficiency in FPGA-based deep learning accelerators", Journal of Systems Architecture, Vol. 163, Page 103410, 2025.
  7. Brar, D.S., Singh, B., Nanda, V., "Effect of optimisations on CNN performance in identifying adulteration of red chilli powder", Microchemical Journal, Vol. 215, Page 114231, 2025.
  8. Adithya, M.S., Brindha, G., "Distributed deep learning training on PARAM Seva: empirical analysis of data parallelism with PyTorch DDP", Proceedings of the 2026 International Conference on AI-Driven Smart Systems and Ubiquitous Computing (ICAUC 2026), Pages 1979-1986, 2026, Thailand.
  9. Kataria, J., Panda, S.P., “Smart Technologies and Intelligent Computing”, Pages 252-258, CRC Press, London, 2026.
  10. Le, H., Deng, W., Nguyen, K.T.P., Medjaher, K., Gogu, C., Wu, D., "Physics-informed transfer learning by embedding physics into activation functions: an application in battery health management", Applied Energy, Vol. 406, Page 127161, 2026.
  11. Khademi, A., Dufour, S., "Physics-informed neural networks with trainable sinusoidal activation functions for approximating the solutions of the Navier-Stokes equations", Computer Physics Communications, Vol. 314, Page 109672, 2025.
  12. Sayın, K.A., Gürsoy, N.K., Yolcu, T., Gürsoy, A., "On the synergy of optimizers and activation functions: a CNN benchmarking study", Mathematics, Vol. 13, Issue 13, Page 2088, 2025.
  13. Al-Mudhafar, W.J., Abbas, M.A., Watheq, B., Al-Mudhafar, J., "DoEHypOpt: an efficient approach of hyperparameter optimization in machine learning using design of experiments applied on multivariate permeability modeling", Mathematical Geosciences, Vol. 58, Issue 2, Pages 403-439, 2025.
  14. Brajevic, I., Tuba, E., Tuba, M., "Ensemble learning with systematic hyperparameter optimization for urban-bike-sharing demand prediction", Sustainability, Vol. 18, Issue 8, Page 3766, 2026.
  15. Söylemez, Ö.F., "The role of weight initialization and preprocessing techniques in analyzing chest X-ray images with deep neural networks: a comparative study", Dicle Üniversitesi Mühendislik Fakültesi Mühendislik Dergisi, Vol. 17, Issue 1, Page 1, 2026.
  16. Erden, C., "Adaptive learning rate optimization in deep recurrent architectures for precision PM2.5 forecasting under climate variability", Chemosphere, Vol. 397, Page 144875, 2026.