PREDICTIVE PERFORMANCES OF IMPLICITLY AND EXPLICITLY ROBUST CLASSIFIERS ON HIGH DIMENSIONAL DATA

Abstract

The goal of this paper is to demonstrate via extensive simulation that implicit robustness can substantially outperform explicit robust inthe pattern recognition of contaminated high dimension low sample size data.Our work speciŞcally demonstrates via extensive computational simulationsand applications to real life data, that random subspace ensemble learning machines, although not explicitly structurally designed as a robustness-inducingsupervised learning paradigms, outperforms the structurally robustness-seekingclassiŞers on high dimension low sample size datasets. Random forest (RF),which is arguably the most commonly used random subspace ensemble learningmethod, is compared to various robust extensions/adaptations of the discriminant analysis classiŞer, and our work reveals that RF, although not inherentlydesigned to be robust to outliers, substantially outperforms the existing techniques speciŞcally designed to achieve robustness. SpeciŞcally, by exploringdifferent scenarios of the sample size n and the input space dimensionality palong with the corresponding capacity ? = n/p with ? < 1, we demonstratethrough extensive simulations that regardless of the contamination rate ?, RFpredictively outperforms the explicitly robustness-inducing classiŞcation techniques when the intrinsic dimensionality of the data is large.