10.1162/153244301753683717
Applied Physics LettersPublished 1 January 2000Open access
Citations107
SJR quartileQ1
SJR score0.90
SNIP1.00
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is found that Bayes point machines consistently outperform support vector machines on both surrogate data and real-world benchmark data sets and it is demonstrated that the real-valued output of single Bayes points on novel test points is a valid confidence measure and leads to a steady decrease in generalisation error when used as a rejection criterion.
Abstract
This DOI is not currently attached to any metadata records. DOIs can’t actually ever be deleted (they’re persistent), but sometimes our members create DOIs in error. We do have a process to approximate deletion which we follow only in rare cases where the DOI has been genuinely created in error, and most crucially, if the DOI has never been published anywhere online or in print and never otherwise distributed to or communicated with anyone (authors, readers, reviewers, etc.
Keywords
Arts and HumanitiesHealth Professions
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
Journal of the Franklin InstituteAn introduction to probability theory and its applications
29,966 Citations1958
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Choice Reviews OnlineNeural networks for pattern recognition
18,687 Citations1994
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
A training algorithm for optimal margin classifiers
11,594 Citations1992Bernhard E. Boser, Isabelle Guyon +1 more
A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented, applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions.
Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features
7,925 Citations1998Thorsten Joachims
SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.
The MIT Press eBooksFast Training of Support Vector Machines Using Sequential Minimal Optimization
5,462 Citations1998John Platt
Journal of the American Statistical AssociationSpline Models for Observational Data.
5,025 Citations1991Hans‐Georg Müller, Grace Wahba
Lecture notes in statisticsBayesian Learning for Neural Networks
4,375 Citations1996Radford M. Neal
Bayesian Learning for Neural Networks shows that Bayesian methods allow complex neural network models to be used without fear of the "overfitting" that can occur with traditional neural network learning methods.
Technical reportsMaking Large-Scale SVM Learning Practical
4,317 Citations2006Thorsten Joachims
This chapter presents algorithmic and computational results developed for SVM light V 2.0, which make large-scale SVM training more practical and give guidelines for the application of SVMs to large domains.
ePrints Soton (University of Southampton)An Introduction to Support Vector Machines
4,120 Citations2000Nello Cristianini, John Shawe‐Taylor
Neural ComputationApproximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
3,654 Citations1998Thomas G. Dietterich
This article reviews five approximate statistical tests for determining whether one learning algorithm outperforms another on a particular learning task and measures the power (ability to detect algorithm differences when they do exist) of these tests.
Philosophical Transactions of the Royal Society of London Series A Containing Papers of a Mathematical or Physical CharacterXVI. Functions of positive and negative type, and their connection the theory of integral equations
1,980 Citations1909James W. Mercer
Lecture notes in computer scienceA Generalized Representer Theorem
1,587 Citations2001Bernhard Schölkopf, Ralf Herbrich +1 more
The result shows that a wide range of problems have optimal solutions that live in the finite dimensional span of the training examples mapped into feature space, thus enabling us to carry out kernel algorithms independent of the (potentially infinite) dimensionality of the feature space.
Machine LearningSoft Margins for AdaBoost
1,306 Citations2001Gunnar Rätsch, Takashi Onoda +1 more
It is found that ADABOOST asymptotically achieves a hard margin distribution, i.e. the algorithm concentrates its resources on a few hard-to-learn patterns that are interestingly very similar to Support Vectors.
American Journal of PhysicsProbability, Frequency and Reasonable Expectation
1,293 Citations1946R. T. Cox
IEEE Transactions on Neural NetworksInput space versus feature space in kernel-based methods
1,166 Citations1999Bernhard Schölkopf, Mika Sirén +5 more
The geometry of feature space is reviewed, and the connection between feature space and input space is discussed by dealing with the question of how one can, given some vector in feature space, find a preimage in input space.
The MIT Press eBooksProbabilities for SV Machines
1,015 Citations2000John C. Platt
This chapter contains sections titled: Introduction, Fitting a Sigmoid After the SVM, Empirical Tests, Conclusions, Appendix: Pseudo-code for the Sigmoids Training.
IEEE Transactions on Pattern Analysis and Machine IntelligenceBayesian classification with Gaussian processes
748 Citations1998Christopher K. I. Williams, David Barber
A Bayesian treatment is provided, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the necessary integration over y is carried out using Laplace's approximation, and the method is generalized to multiclass problems (m>2) using the softmax function.
DSpace@MIT (Massachusetts Institute of Technology)Support Vector Machines: Training and Applications
704 Citations1997E. Osuna, Robert M. Freund +1 more
Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond
639 Citations1998Christopher K. I. Williams
The main aim of this paper is to provide a tutorial on regression with Gaussian processes, starting from Bayesian linear regression, and showing how by a change of viewpoint one can see this method as a Gaussian process predictor based on priors over functions, rather than on prior over parameters.
Bayesian methods for adaptive models
629 Citations1992David Mackay
The Bayesian framework for model comparison and regularisation is demonstrated by studying interpolation and classification problems modelled with both linear and non-linear models, and it is shown that the careful incorporation of error bar information into a classifier's predictions yields improved performance.
Machine LearningTraining Invariant Support Vector Machines
582 Citations2002Dennis DeCoste, Bernhard Schölkopf
IEEE Transactions on Information TheoryStructural risk minimization over data-dependent hierarchies
527 Citations1998John Shawe‐Taylor, Peter L. Bartlett +2 more
A result is presented that allows one to trade off errors on the training sample against improved generalization performance, and a more general result in terms of "luckiness" functions, which provides a quite general way for exploiting serendipitous simplicity in observed data to obtain better prediction accuracy from small training sets.
Some PAC-Bayesian theorems
489 Citations1998David McAllester
The MIT Press eBooksLearning Kernel Classifiers
456 Citations2001Ralf Herbrich
This book provides the first comprehensive overview of both the theory and algorithms of kernel classifiers, including the most recent developments, and a detailed introduction to learning theory, including VC and PAC-Bayesian theory, data-dependent structural risk minimization, and compression bounds.
PAC-Bayesian model averaging
372 Citations1999David McAllester
The method constructs an optimized weighted mixture of concepts analogous to a Bayesian posterior distribution, and the main result is stated for bounded loss, a preliminary analysis for unbounded loss is also given.
International Joint Conference on Artificial IntelligenceVersion spaces: a candidate elimination approach to rule learning
336 Citations1977Tom M. Mitchell
An approach to this problem is presented which is guaranteed to find, without backtracing, all rule versions consistent with a set of positive and negative training instances.
BiometrikaTHE ESTIMATION OF THE LOCATION AND SCALE PARAMETERS OF A CONTINUOUS POPULATION OF ANY GIVEN FORM
255 Citations1939E. J. G. Pitman
Neural ComputationGaussian Processes for Classification: Mean-Field Algorithms
215 Citations2000Manfred Opper, Ole Winther
A mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems is derived and an approximate leave-one-out estimator for the generalization error is computed.
Physical Review LettersGeneralization performance of Bayes optimal classification algorithm for learning a perceptron
152 Citations1991Manfred Opper, David Haussler
It is shown that approximations to the generalization error of the Bayes optimal algorithm can be achieved by learning algorithms that use a two-layer neutral net to learn a perceptron.
Journal of Physics A Mathematical and GeneralOn the ability of the optimal perceptron to generalise
120 Citations1990Manfred Opper, Wolfgang Kinzel +2 more
A linearly separable Boolean function is derived from a set of examples by a perceptron with optimal stability by calculating the probability to reconstruct a pattern which is not learnt using the replica method.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Two Papers on the Comparison of Bayesian and Frequentist Approaches to Statistical Problems of Prediction: Bayesian Tolerance Regions
73 Citations1964J. Aitchison
Probabilistic Methods for Support Vector Machines
64 Citations1999Peter Sollich
A framework for interpreting Support Vector Machines (SVMs) as maximum a posteriori (MAP) solutions to inference problems with Gaussian Process priors is described, which can provide intuitive guidelines for choosing a 'good' SVM kernel.
Europhysics Letters (EPL)Optimal Learning with a Neural Network
60 Citations1993Timothy L. H. Watkin
It is found that the optimally-trained spherical perceptron may learn a linearly-separable rule as well as any possible network as well as any possible network in a neural network.
A PAC-Bayesian Margin Bound for Linear Classifiers: Why SVMs work
58 Citations2000Ralf Herbrich, Thore Graepel
A bound on the generalisation error of linear classifiers in terms of a refined margin quantity on the training set is presented in a PAC-Bayesian framework and is based on geometrical arguments in the space oflinear classifiers.
Neural ComputationPlaying Billiards in Version Space
55 Citations1997P. Ruján
A ray-tracing method inspired by ergodic billiards is used to estimate the theoretically best decision rule for a given set of linear separable examples, which indicates the billiard estimate of the single Perceptron with best average generalization probability.
UR Research (University of Rochester)Prediction of Generalization Ability in Learning Machines
45 Citations1994Corinna Cortes
The main goal of the dissertation is to merge theory and practice: to develop theoretically based, but experimentally adapted tools that allow an accurate prediction of the generalization error of an arbitrarily complex classifier.
Bayes Point Machines: Estimating the Bayes Point in Kernel Space
45 Citations1999Ralf Herbrich, Th Graepel +1 more
Experimental results are presented indicating that Bayes Point Machines consistently outperform Support Vector Machines in estimating the center of mass by averaging over the trajectory of a billiard ball bouncing in version space.
Bayesian Transduction
27 Citations1999Thore Graepel, Ralf Herbrich +1 more
Experimental results on real world data indicate that Bayesian Transduction compares favourably to the well-known Support Vector Machine, in particular if the posterior probability of labellings is used as a confidence measure to exclude test points of low confidence.
The Kernel Gibbs Sampler
21 Citations2000Thore Graepel, Ralf Herbrich
An algorithm that samples the hypothesis space of kernel classifiers with a uniform prior over normalised weight vectors and a likelihood based on a model of label noise leads to a piecewise constant posterior that can be sampled by the kernel Gibbs sampler (KGS).
The MIT Press eBooksComputing the Bayes Kernel Classifier
16 Citations2000P. Ruján, Mario Marchand
This chapter contains sections titled: Introduction, A Simple Geometric Problem, The Maximal Margin Perceptron, The Bayes Perceptrons, The Kernel-Billiard, Numerical Tests, Conclusions, Appendix.
Robust Bayes Point Machines
13 Citations2000Ralf Herbrich, Th Graepel +1 more
This work introduces a technique that allows the BPM to construct hypotheses with non{zero training error similar to soft margin SVMs with quadratic penelisation of the margin slacks, and shows that BPMs outperform hard margin Support Vector Machines (SVMs) on real world datasets.
Sparsity vs. Large Margins for Linear Classifiers
7 Citations2000Ralf Herbrich, Thore Graepel +1 more
This work provides small sample size bounds on the generalisation error of linear classifiers that take advantage of large observed margins on the training set and sparsity in the data dependent expansion coefficients to demonstrate that they can be combined.