SavedAdd your journal

Journals / Uludağ Üniversitesi Mühendislik Fakültesi Dergisi / 2021 / Cilt: 26 - Sayı: 3

A Sequential Minimal Optimization Algorithm with Second-Order like Information to solve a Non-Smooth Support Vector Regression Constrained Dual Problem

DÜZGÜN OLMAYAN DESTEK VEKTÖR REGRESYONU KISITLI İKİNCİL PROBLEMİNİ ÇÖZMEK İÇİN İKİNCİ DERECEDEN BENZER BİLGİLERE SAHİP BİR ARDIŞIK ASGARİ ENİYİLEME ALGORİTMASI

Aykut KOCAOĞLU
Pages
1111–1120
Publication Date
2022-11-06
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Additionally, the results are compared with the classic smooth SVR problem.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The ε-insensitive Support Vector Regression (ε-SVR) is expressed by the ε-insensitive regularized 𝑙1 error loss function, and has the robustness of the 𝑙1 loss function, as well as being insensitive to small errors.
📊
Key Findings & Data:In the proposed WSS, second-order like information is employed and its superiority over the first-order counterpart to solve this non smooth optimization problem has been demonstrated by the results obtained on a set of real-world datasets.
💡
Conclusion & Impact:Additionally, the results are compared with the classic smooth SVR problem.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Destek vektör regresyonu · Düzgün olmayan eniyileme · Çalışma kümesi seçimi

Abstract

The ε-insensitive Support Vector Regression (ε-SVR) is expressed by the ε-insensitive regularized 𝑙1 error loss function, and has the robustness of the 𝑙1 loss function, as well as being insensitive to small errors. Also, the regularization of error loss provides a control over the flatness of the solution. In this study, the ε-SVR dual problem is derived as a non-smooth convex piecewise quadratic problem under the equality and inequality constraints, with the advantage of having half the optimization variables of the classical smooth SVR dual problem. This convex non-smooth dual optimization problem is solved by an efficient Sequential Minimal Optimization (SMO) algorithm using a working set selection (WSS) based on minimizing an upper bound for the difference between consecutive loss function values. Previously, in the solution of the non-smooth dual ε-SVR problem with SMO algorithm, first order information was used by selecting the pairs in a manner that violates the Karush-Kuhn-Tucker conditions the most. In the proposed WSS, second-order like information is employed and its superiority over the first-order counterpart to solve this non smooth optimization problem has been demonstrated by the results obtained on a set of real-world datasets. Additionally, the results are compared with the classic smooth SVR problem.

Özet

ε-duyarsız Destek Vektör Regresyonu (ε-DVR), ε-duyarsızlık özelliğine sahip düzenlenmiş 𝑙1 hata kayıp fonksiyonu ile ifade edilir ve 𝑙1 kayıp fonksiyonunun sahip olduğu gürbüz olma özelliği yanında küçük hatalara karşı duyarsız olma özelliğine de sahiptir. Ayrıca, düzenlenmiş hata ile çözümün düzlüğü üzerinde kontrol sağlanır. Bu çalışmada, ε-DVR ikincil problemi, klasik pürüzsüz DVR ikincil probleminin yarısı kadar eniyileme değişkenine sahip olma avantajıyla eşitlik ve eşitsizlik kısıtları altında düzgün olmayan dışbükey parçalı ikinci dereceden problem olarak türetilmiştir. Türetilen bu dışbükey düzgün olmayan ikincil eniyileme problemi, ardışık kayıp fonksiyonu değerleri arasındaki farka ilişkin bir üst sınırın en aza indirilmesine dayanan bir çalışma kümesi seçimi (ÇKS) kullanan verimli bir Ardışık Asgari Eniyileme (AAE) algoritması ile çözülmüştür. Daha önce düzgün olmayan ikincil ε-DVR probleminin AAE algoritması ile çözümünde ÇKS için Karush-KuhnTucker (KKT) koşullarını en fazla ihlal eden çiftler alınarak birinci dereceden bilgiler kullanılmıştır. Önerilen ÇKS’de ise ikinci dereceden benzer bilgiler kullanılmaktadır ve bu düzgün olmayan eniyileme problemini çözmek için birinci dereceden emsaline göre üstünlüğü bir dizi gerçek dünya veri kümesi üzerinde elde edilen sonuçlarla gösterilmiştir. Ayrıca, sonuçlar klasik pürüzsüz DVR ile de karşılaştırılmıştır

🔖

Cite This Article

KOCAOĞLU, A. (2022). A Sequential Minimal Optimization Algorithm with Second-Order like Information to solve a Non-Smooth Support Vector Regression Constrained Dual Problem. Uludağ Üniversitesi Mühendislik Fakültesi Dergisi, Cilt: 26 - Sayı: 3, 1111–1120. https://ajindex.com/articles/643741

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (2)

  1. Abe, S. (2015) Optimizing working sets for training support vector regressors by Newton’s method, International joint conference on neural networks, IJCNN, Killarney, Ireland. doi:10.1109/IJCNN.2015.7280309 2. Abe, S. (2016) Fusing sequential minimal optimization and Newton’s method for support vector training, International Journal of Machine Learning and Cybernetics, 7(3), 345–364. doi:10.1007/s13042-014-0265-x 3. Barbero, A., Lopez, J, ve Dorronsoro, J.R. (2009) Cycle-breaking acceleration of SVM training, Neurocomputing, 72(7–9), 1398–1406. doi:10.1016/j.neucom.2008.12.014 4. Barbero, A. ve Dorronsoro, J.R. (2011) Momentum sequential minimal optimization: an accelerated method for support vector machine training, International joint conference on neural networks, IJCNN. doi:10.1109/IJCNN.2011.6033245 5. Boser, B., Guyon, I. ve Vapnik, V. (1992) A training algorithm for optimal margin classifiers, Proceedings of the fifth annual workshop on Computational Learning Theory, 144–152. doi:10.1145/130385.130401 6. Bottou, L. ve Lin C.J. (2007) Support vector machine solvers in Large scale kernel machines, MIT Press, Cambridge. 7. Cortes, C. ve Vapnik, V. (1995)
  2. Support-vector network, Machine Learning, 20, 273–297. 8. Fan, R.E., Chen, P.H. ve Lin, C.J. (2005) Working set selection using second order information for training support vector machines, Journal of Machine Learning Research, 6, 1889–1918. 9. Flake, G.W. ve Lawrence, S. (2001) Efficient SVM regression training with SMO, Machine Learning, 46(1), 271–290. doi:10.1023/A:1012474916001 10. Guo, J., Takahashi, N. ve Nishi, T. (2006) A novel sequential minimal optimization algorithm for support vector regression, Lecture Notes in Computer Science, 4232, 827–836. doi:10.1007/11893028_92 11. Keerthi, S.S., Shevade, S.K., Bhattacharyya, C. ve Murthy, K.R.K. (2001) Improvements to Platt’s SMO algorithm for SVM classifier design, Neural Computation, 13(3), 637–649. doi:10.1162/089976601300014493 12. Kocaoğlu, A. (2019) An Efficient SMO Algorithm for Solving Non-smooth Problem Arising in ε-Insensitive Support Vector Regression, Neural Processing Letters, 50, 933–955. doi:10.1007/s11063-018-09975-3 13. Kumar, B., Sinha, A., Chakrabarti, S. ve Vyas, O. P. (2021) A fast learning algorithm for One-Class Slab Support Vector Machines, arXiv: 2011.03243. https://arxiv.org/abs/2011.03243 doi:10.1016/j.knosys.2021.107267 14. Lin, C.J. (2001) On the convergence of the decomposition method for support vector machines, I