Support Vector Machines are Universally Consistent

It is shown that the 1-norm soft margin classifier with Gaussian RBF kernel on a compact subset X of Rd and regularization parameter cn = nβ-1 is universally consistent, if n is the training set size and 0 >β> 1/d.

Abstract

We show that support vector machines of the 1-norm soft margin type are universally consistent provided that the regularization parameter is chosen in a distinct manner and the kernel belongs to a specific class—the so-called universal kernels—which has recently been considered by the author. In particular it is shown that the 1-norm soft margin classifier with Gaussian RBF kernel on a compact subset X of Rd and regularization parameter cn=nβ−1 is universally consistent, if n is the training set size and 0<β<1/d.