We show that various classifiers that are based on a minimization of a regularized risk are universally consistent, i.e.~they can asymptotically learn in every classification task. In particular we focus on the role of the loss functions used in these algorithms. As an application of our general framework, several types of support vector machines as well as regularization networks are treated. Our methods combine techniques from stochastics, approximation theory and functional analysis.
I. Steinwart, Consistency of Support Vector Machines and other Regularized Kernel Machines. IEEE Transactions on Information Theory, Vol. 51, pp. 128-142, 2005. [ Abstract ]
