Journals / Erciyes Üniversitesi Fen Bilimleri Enstitüsü Fen Bilimleri Dergisi / 2026 / Cilt: 42 Sayı: 3

One-Sample Tilted Empirical Risk Minimization: Finite-Sample Tracking, Denominator Order, and Robustness

Tek Örnekli Eğilmiş Ampirik Risk Minimizasyonu: Sonlu Örneklem İzlemesi, Payda Sırası ve Sağlamlık

Zafer Doğan(KOÇ ÜNİVERSİTESİ)0000-0002-5078-4590
Pages
—
Publication Date
2026-10-28
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The evidence is empirical and finite-horizon.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Background/Aim: Classical tilted empirical risk minimization (TERM) assigns loss-dependent weights through a global normalizer.
📊
Key Findings & Data:We ask how closely a strict one-current-sample approximation tracks finite-sample TERM and how its finite-step behavior depends on denominator order, normalizer rate, and tilt.
💡
Conclusion & Impact:The evidence is empirical and finite-horizon.
🔬Design:Araştırma Makalesi
📍Scope:Comprehensive Sample
⚙️Variables:Eğilmiş Ampirik Risk Minimizasyonu · Stokastik Yaklaşım · Çevrim İçi Öğrenme

Abstract

Background/Aim: Classical tilted empirical risk minimization (TERM) assigns loss-dependent weights through a global normalizer. Direct evaluation on one observation removes relative weighting. We ask how closely a strict one-current-sample approximation tracks finite-sample TERM and how its finite-step behavior depends on denominator order, normalizer rate, and tilt. Methods: A scalar moving-average normalizer carries cross-sample information between updates. We compared a previous-state denominator with a matched current-sample-first order on a toy problem using full-batch oracles, seven normalizer rates, and 20 shared streams. Scale-calibrated tilts were then evaluated with Polyak–Ruppert-averaged stochastic gradient descent (SGD) under four Breast Cancer Wisconsin Diagnostic (WDBC) label-noise conditions and seven Abalone response-contamination conditions. Results: Both orders remained close to the toy finite-sample references. They were exactly identical at zero tilt; at nonzero tilt, their local low-rate discrepancy was approximately proportional to the product of learning and normalizer rates and changed direction with tilt. No order was uniformly superior. Negative tilt improved on empirical risk minimization (ERM) in specified corrupted conditions. Representative task-specific baselines were stronger in the most corrupted conditions, while the finite-sample TERM tracking findings were unchanged. Conclusion: One persistent scalar can preserve tilt-dependent one-sample updates and closely track classical TERM in the tested finite-dimensional settings. The evidence is empirical and finite-horizon.

Özet

Arka Plan/Amaç: Klasik eğilmiş ampirik risk minimizasyonu (TERM), eğitim gözlemlerinin tümü üzerinden hesaplanan bir normalleştiriciyle kayba bağlı ağırlıklar atar. Tek gözlem üzerinde doğrudan değerlendirme bu göreli ağırlıklandırmayı ortadan kaldırır. Katı, tek-güncel-örnekli ve durum tutan bir yaklaşımın sonlu örneklem TERM’ini ne ölçüde izlediğini; sonlu-adım davranışının payda sırasına, normalleştirici hızına ve eğime nasıl bağlı olduğunu inceliyoruz. Yöntemler: Skaler üstel hareketli-ortalama normalleştirici, stokastik güncellemeler arasında örnekler-arası bilgiyi taşır. Önceki-durum paydasını, tam-yığın referanslar, yedi normalleştirici hızı ve 20 ortak örnek akışıyla doğrusal-regresyon oyuncak probleminde eşleştirilmiş güncel-örnek-öncelikli sırayla karşılaştırdık. Ölçek-kalibreli negatif, sıfır ve pozitif eğimler daha sonra dört Breast Cancer Wisconsin Diagnostic (WDBC) etiket-gürültüsü ve yedi Abalone yanıt-kirlenmesi koşulunda Polyak–Ruppert ortalamalı stokastik gradyan inişi (SGD) ile değerlendirildi. Bulgular: İki sıra da oyuncak sonlu-örneklem referanslarına yakın kaldı. Sıfır eğimde tam olarak özdeşti; sıfır-dışı eğimde yerel düşük-hız farkı yaklaşık olarak öğrenme ve normalleştirici hızlarının çarpımıyla orantılıydı ve yönü eğime göre değişti. Hiçbir sıra her durumda üstün değildi. Negatif eğim, belirtilen kirlenmiş koşullarda ampirik risk minimizasyonunu (ERM) iyileştirdi. Göreve özgü temsili yöntemler en yüksek kirlenme koşullarında daha güçlüydü; sonlu-örneklem TERM izleme bulguları ise değişmedi. Sonuç: Kalıcı tek bir skaler, eğime bağlı tek-örnekli güncellemeleri koruyabilir ve incelenen sonlu boyutlu ortamlarda klasik TERM’i yakından izleyebilir. Kanıtlar ampirik ve sonlu ufukludur.

🔖

Cite This Article

Doğan, Z. (2026). One-Sample Tilted Empirical Risk Minimization: Finite-Sample Tracking, Denominator Order, and Robustness. Erciyes Üniversitesi Fen Bilimleri Enstitüsü Fen Bilimleri Dergisi, Cilt: 42 Sayı: 3. https://doi.org/10.65520/erciyesfen.1957213

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (24)

  1. Wang, M., Fang, E. X., and Liu, H. Stochastic compositional gradient descent: Algorithms for minimizing compositions of expected-value functions. Mathematical Programming, 161(1–2), 419–449, 2017.
  2. Wang, B. and Yang, T. Finite-sum coupled compositional stochastic optimization: Theory and applications. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 23292–23317. PMLR, 2022.
  3. Wang, B. and Yang, T. A near-optimal single-loop stochastic algorithm for convex finite-sum coupled compositional optimization. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 65091–65121. PMLR, 2025.
  4. Li, T., Beirami, A., Sanjabi, M., and Smith, V. On tilted losses in machine learning: Theory and applications. Journal of Machine Learning Research, 24(142), 1–79, 2023.
  5. Rockafellar, R. T. and Uryasev, S. Optimization of conditional value-at-risk. The Journal of Risk, 2(3), 21–41, 2000.
  6. Duchi, J. C. and Namkoong, H. Learning models with uniform performance via distributionally robust optimization. The Annals of Statistics, 49(3), 1378–1406, 2021.
  7. Aminian, G., Asadi, A. R., Li, T., Beirami, A., Reinert, G., and Cohen, S. N. Generalization and robustness of the tilted empirical risk. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 1419–1461. PMLR, 2025.
  8. Wei, X., Zhou, L., Wang, B., Lin, C.-J., and Yang, T. A geometry-aware efficient algorithm for compositional entropic risk minimization, 2026. arXiv preprint; accepted to the 43rd International Conference on Machine Learning, final proceedings metadata not yet available.
  9. Yıldırım, Y. E., Demir, S., and Doğan, Z. Benefits of online tilted empirical risk minimization: A case study of outlier detection and robust regression. In 2025 IEEE 35th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6. IEEE, 2025.
  10. Borkar, V. S. Stochastic approximation with two time scales. Systems & Control Letters, 29(5), 291–294, 1997.
  11. Borkar, V. S. Stochastic Approximation: A Dynamical Systems Viewpoint, volume 48 of Texts and Readings in Mathematics. Springer Nature Singapore, Singapore, 2nd edition, 2023.
  12. Kushner, H. J. and Yin, G. G. Stochastic Approximation and Recursive Algorithms and Applications, volume 35 of Stochastic Modelling and Applied Probability. Springer New York, New York, 2nd edition, 2003.
  13. Huber, P. J. Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73–101, 1964.
  14. Catoni, O. Challenging the empirical mean and empirical variance: A deviation study. Annales de l’Institut Henri Poincaré, Probabilités et Statistiques, 48(4), 1148–1185, 2012.
  15. Brownlees, C., Joly, E., and Lugosi, G. Empirical risk minimization for heavy-tailed losses. The Annals of Statistics, 43(6), 2507–2536, 2015.
  16. Natarajan, N., Dhillon, I. S., Ravikumar, P. K., and Tewari, A. Learning with noisy labels. In Advances in Neural Information Processing Systems, volume 26, pages 1196–1204, 2013.
  17. Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., and Qu, L. Making deep neural networks robust to label noise: A loss correction approach. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1944–1952. IEEE, 2017.
  18. Ma, X., Huang, H., Wang, Y., Romano, S., Erfani, S., and Bailey, J. Normalized loss functions for deep learning with noisy labels. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 6543–6553. PMLR, 2020.
  19. Zhu, Y., Zhang, J., Gangrade, A., and Scott, C. Label noise: Ignorance is bliss. In Advances in Neural Information Processing Systems, volume 37, 2024.
  20. Guo, H., Yi, G. Y., and Wang, B. Learning from noisy labels via conditional distributionally robust optimization. In Advances in Neural Information Processing Systems, volume 37, 2024.
  21. Wolberg, W., Mangasarian, O., Street, N., and Street, W. Breast cancer wisconsin (diagnostic). UCI Machine Learning Repository, 1993. Dataset.
  22. Nash, W., Sellers, T., Talbot, S., Cawthorn, A., and Ford, W. Abalone. UCI Machine Learning Repository, 1994. Dataset.
  23. Polyak, B. T. and Juditsky, A. B. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30(4), 838–855, 1992.
  24. Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.