We investigate statistical properties for a broad class of modern kernel based regression (KBR) methods. These kernel methods were developed during the last decade and are inspired by convex risk minimization in infinite dimensional Hilbert spaces. One leading example is support vector regression. We first describe the relation between the used loss function $L$ of the KBR method and the tail of the response variable. We then establish the $L$-risk consistency for KBR which gives the mathematical justification for the statement that these methods are able to 'learn'. Then we consider robustness properties of such kernel methods. In particular, our results allow to choose the loss function and the kernel to obtain computational tractable and consistent KBR methods having bounded influence functions. Furthermore, bounds for the sensitivity curve which is a finite sample version of the influence function are developed, and the relationship between KBR and classical M-estimators is discussed.
A. Christmann and I. Steinwart, Consistency and Robustness of Kernel Based Regression. Bernoulli, Vol. 13, pp. 799-819, 2007. Los Alamos National Laboratory Technical Report LA-UR-04-8797. [ Abstract | PDF (395 KB) ]
