Journals / Konya Journal of Engineering Sciences / 2022 / Cilt: 10 Sayı: 3

A SYSTEMATIC AND EFFICIENT INPUT SELECTION METHOD FOR ARTIFICIAL NEURAL NETWORKS USING MIXED-INTEGER NONLINEAR PROGRAMMING

Kesikli ve Sürekli Optimizasyon Kullanarak Yapay Sinir Ağları için Sistematik Girdi Seçimi Yöntemi

Pages
762–773
Publication Date
2022-09-01
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Results show that better test performance can be achieved with optimally selected inputs, resulting in reduced overfitting.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Selection of input variables of the empirical models has vital effect on the prediction performance, reduced overfitting and reduced computational load.
📊
Key Findings & Data:Such selection uses binary (0-1) variables to represent the presence of the input variables and trains traditional continuous network weights simultaneously.
💡
Conclusion & Impact:Results show that better test performance can be achieved with optimally selected inputs, resulting in reduced overfitting.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Girdi Seçimi · Yapay Sinir Ağları · Kesikli ve sürekli optimizasyon

Abstract

Selection of input variables of the empirical models has vital effect on the prediction performance, reduced overfitting and reduced computational load. Various trials and error and sequential methods in the literature to deal with input selection for artificial neural networks (ANNs). However, these methods are not considered as automatic and systematic. This study proposes a novel and efficient mixed integer nonlinear programming-based approach to handle optimal input selection and the ANN training simultaneously for classification problems. Such selection uses binary (0-1) variables to represent the presence of the input variables and trains traditional continuous network weights simultaneously. Two classification case studies are given to demonstrate the advantages by using widely used data sets and statistical measures. The first data set is related to the characterization of the type of a tumor related to breast cancer, the second data set is about predicting the type of a biotechnological product using different features, the last one is related to heart failure prediction. Results show that better test performance can be achieved with optimally selected inputs, resulting in reduced overfitting. The proposed approach delivers a significant advantage during the design and training of the ANNs and is also applicable to other empirical models.

Özet

Ampirik modellerin girdi değişkenlerinin seçimi, tahmin performansı, azaltılmış fazla uydurma ve hesaplama yükünün azaltılması üzerinde önemli etkiye sahiptir. Literatürde yapay sinir ağları (YSA) için girdi seçimi ile ilgili çeşitli deneme yanılma yöntemleri mevcuttur ancak bu metodlar sistematik ve otomatik olarak kabul edilmemektedir. Bu çalışma, sınıflandırma problemleri için optimal girdi seçimi ve YSA eğitimini aynı anda ele almak için yeni ve verimli bir karma tamsayılı doğrusal olmayan programlama tabanlı bir yaklaşım önermektedir. Bu seçim, girdi değişkenlerinin varlığını temsil etmek için ikili (0-1) değişkenleri kullanır ve geleneksel sürekli ağ ağırlıklarını veya parametrelerini aynı anda eğitir. Yaygın olarak kullanılan veri setleri ve istatistiksel ölçümler kullanarak avantajları göstermek amacıyla üç sınıflandırma vaka çalışması sunulmuştur. Birinci veri seti meme kanseri ile ilgili tümörün tipin-in karakterizasyonu ile ilgili olup, ikinci veri seti ise farklı özellikler kullanılarak bir biyoteknolojik ürünün tipinin tahmin edilmesi ile ilgilidir, son veri seti ise kalp sağlığı ile ilgilidir. Sonuçlar, optimal olarak seçilen girdiler ile düşük fazla uydurma sayesinde daha iyi test performansının elde edilebileceğini göstermektedir. Önerilen yaklaşım, YSA'ların tasarımı ve eğitimi sırasında önemli bir avantaj sağlar ve diğer ampirik modellere de uygulanabilir.

🔖

Cite This Article

ŞILDIR, H., AYDIN, E. (2022). A SYSTEMATIC AND EFFICIENT INPUT SELECTION METHOD FOR ARTIFICIAL NEURAL NETWORKS USING MIXED-INTEGER NONLINEAR PROGRAMMING. Konya Journal of Engineering Sciences, Cilt: 10 Sayı: 3, 762–773. https://ajindex.com/articles/908986

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (12)

  1. Agarap, Abien Fred M. 2018. “On Breast Cancer Detection: An Application of Machine Learning Algorithms on the Wisconsin Diagnostic Dataset.” In Proceedings of the 2nd International Conference on Machine Learning and Soft Computing, 5–9.
  2. Aha, David W, and Richard L Bankert. 1996. “A Comparative Evaluation of Sequential Feature Selection Algorithms.” In Learning from Data, Springer, 199–206.
  3. Akdag, Unal, M. Aydin Komur, and A. Feridun Ozguc. 2009. “Estimation of Heat Transfer in Oscillating Annular Flow Using Artifical Neural Networks.” Advances in Engineering Software 40(9): 864–70.
  4. Alom, Md Zahangir et al. 2019. “A State-of-the-Art Survey on Deep Learning Theory and Architectures.” Electronics (Switzerland) 8(3): 292. Azadeh, A., S. F. Ghaderi, and S. Sohrabkhani. 2008. “Annual Electricity Consumption Forecasting by Neural Network in High Energy Consuming Industrial Sectors.” Energy Conversion and Management 49(8): 2272–78.
  5. Benbrahim, Houssam, Hanaâ Hachimi, and Aouatif Amine. 2019. “Comparative Study of Machine Learning Algorithms Using the Breast Cancer Dataset.” In International Conference on Advanced Intelligent Systems for Sustainable Development, Springer, 83–91.
  6. Bredensteiner, Erin J., and Kristin P. Bennett. 1999. “Multicategory Classification by Support Vector Machines.” Computational Optimization and Applications 12(1–3): 53–79.
  7. Castellano, Giovanna, and Anna Maria Fanelli. 2000. “Variable Selection Using Neural-Network Models.” Neurocomputing 31(1–4): 1–13.
  8. Chicco, Davide, and Giuseppe Jurman. 2020. “Machine Learning Can Predict Survival of Patients with Heart Failure from Serum Creatinine and Ejection Fraction Alone.” BMC medical informatics and decision making 20(1): 1–16.
  9. Diaz, G I, A Fokoue-Nkoutche, G Nannicini, and H Samulowitz. 2017. “An Effective Algorithm for Hyperparameter Optimization of Neural Networks.” IBM Journal of Research and Development 61(4/5): 9:1-9:11. Duran, Marco A., and Ignacio E. Grossmann. 1986. “An Outer-Approximation Algorithm for a Class of Mixed-Integer Nonlinear Programs.” Mathematical Programming 36(3): 307–39.
  10. Ferri, Francesc J, Pavel Pudil, Mohamad Hatef, and Josef Kittler. 1994. “Comparative Study of Techniques for Large-Scale Feature Selection.” In Machine Intelligence and Pattern Recognition, Elsevier, 403–13.
  11. Feurer, Matthias, and Frank Hutter. 2019. “Hyperparameter Optimization.” In Automated Machine Learning, Springer, Cham, 3–33.
  12. Hart, William E., Jean Paul Watson, and David L. Woodruff. 2011. “Pyomo: Mo