We establish a general oracle inequality for clipped approximate minimizers of regularized empirical risks and apply this inequality to support vector machine (SVM) type algorithms. We then show that for SVMs using Gaussian RBF kernels for classification this oracle inequality leads to learning rates that are faster than the ones established by Steinwart & Scovel (2004). Finally, we use our oracle inequality to show that a simple parameter selection approach based on a validation set can yield the same fast learning rates without knowing the noise exponents which were required to be known a-priori Steinwart & Scovel (2004).
I. Steinwart, D. Hush, and C. Scovel, An Oracle Inequality for Clipped Regularized Risk Minimizers. In Neural Information Processing Systems 19, pp. 1321-1328, (2007). Los Alamos National Laboratory Technical Report LA-UR-06-3981. [ Abstract | PDF (121 KB) ]
