A Probabilistic Framework for SVM Regression and Error Bar Estimation
Machine LearningPublished 1 January 2002
Junbin Gao, S.R. Gunn, C.J. Harris, Martin Brown
Citations135
SJR quartileQ1
SJR score1.15
SNIP2.14
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper concentrates on the derivation of the evidence and error bar approximation for regression problems and an error bar formula is derived based on the ∈-insensitive loss function.
Abstract
In this paper, we elaborate on the well-known relationship between Gaussian Processes (GP) and Support Vector Machines (SVM) under some convex assumptions for the loss functions. This paper concentrates on the derivation of the evidence and error bar approximation for regression problems. An error bar formula is derived based on the ∈-insensitive loss function.
Keywords
Computer ScienceEngineering
IEEE Transactions on Automatic ControlA new look at the statistical model identification
50,732 Citations1974Hirotugu Akaike
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
International Conference on Neural Information ProcessingAdvances in kernel methods: support vector learning
5,815 Citations1999Bernhard Schölkopf, Christopher J. C. Burges +1 more
Support vector machines for dynamic reconstruction of a chaotic system, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel.
Neural ComputationBayesian Interpolation
4,377 Citations1992David Mackay
The Bayesian approach to regularization and model-comparison is demonstrated by studying the inference problem of interpolating noisy data by examining the posterior probability distribution of regularizing constants and noise levels.
Lecture notes in statisticsBayesian Learning for Neural Networks
4,375 Citations1996Radford M. Neal
Bayesian Learning for Neural Networks shows that Bayesian methods allow complex neural network models to be used without fear of the "overfitting" that can occur with traditional neural network learning methods.
Support Vector Machines for Classification and Regression
2,828 Citations1998S.R. Gunn
IEEE Transactions on Neural NetworksNetwork information criterion-determining the number of hidden units for an artificial neural network model
649 Citations1994Noboru Murata, S. Yoshizawa +1 more
The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike's information criterion (AIC) to be applicable to unfaithful models with general loss criteria including regularization terms.
Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond
639 Citations1998Christopher K. I. Williams
The main aim of this paper is to provide a tutorial on regression with Gaussian processes, starting from Bayesian linear regression, and showing how by a change of viewpoint one can see this method as a Gaussian process predictor based on priors over functions, rather than on prior over parameters.
Neural ComputationAn Equivalence Between Sparse Approximation and Support Vector Machines
505 Citations1998Federico Girosi
If the data are noiseless, the modified version of basis pursuit denoising proposed in this article is equivalent to SVM in the following sense: if applied to the same data set, the two techniques give the same solution, which is obtained by solving the same quadratic programming problem.
Bayesian Interpolation
240 Citations1992David Mackay
Aston Publications Explorer (Aston University)Computing with Infinite Networks
160 Citations1996Christopher K. I. Williams
IEEE Transactions on Neural NetworksModerating the outputs of support vector machine classifiers
155 Citations1999James T. Kwok
Lecture notes in computer scienceNetwork performance assessment for Neurofuzzy data modelling
96 Citations1997Steve R. Gunn, Martin Brown +1 more
This paper evaluates the performance of ten significance measures applied to the problem of determining an appropriate network structure, for data modelling with neurofuzzy systems, with demonstrated advantages with application to both real and synthetic data interpretation problems.
Neural ComputationA Sparse Representation for Function Approximation
84 Citations1998Tomaso Poggio, Federico Girosi
A new general representation for a function is derived as a linear combination of local correlation kernels at optimal sparse locations (and scales) and its relation to principal component analysis, regularization, sparsity principles, and support vector machines is characterized.
Probabilistic Methods for Support Vector Machines
64 Citations1999Peter Sollich
A framework for interpreting Support Vector Machines (SVMs) as maximum a posteriori (MAP) solutions to inference problems with Gaussian Process priors is described, which can provide intuitive guidelines for choosing a 'good' SVM kernel.
DSpace@MIT (Massachusetts Institute of Technology)A Unified Framework for Regularization Networks and Support Vector Machines
62 Citations1999Theodoros Evgeniou, Massimiliano Pontil +1 more
This work presents regularization Networks and Support Vector Machines in a unified framework in the context of Vapnik''s theory of statistical learning which provides a general foundation for the learning problem, combining functional analysis and statistics.
Probabilistic interpretations and bayesian methods for support vector machines
45 Citations1999Peter Sollich
The author shows that a probabilistic interpretation of support vector machines as maximum a posteriori solutions to inference problems with Gaussian process priors and appropriate likelihood functions gives a clear intuitive meaning to SVM kernels, as covariance functions of GP priors.
Approximate learning curves for Gaussian processes
11 Citations1999Peter Sollich
The problem of calculating learning curves of Gaussian processes used for regression is considered, and a simple expression for the generalization error in terms of the eigenvalue decomposition of the covariance function is derived.
Moderating the outputs of support vector machine classifiers
7 Citations2003James T. Kwok
This paper extends the use of moderated outputs to the support vector machine (SVM) by making use of a relationship between SVM and the evidence framework, and the moderated output derived here can be taken as an approximation to the posterior class probability.