SMO Algorithm for Least-Squares SVM Formulations
Neural ComputationPublished 1 February 2003
S. Sathiya Keerthi, Shirish Shevade
Citations119
SJR quartileQ1
SJR score0.83
SNIP1.45
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The well-known SMO algorithm of support vector machines (SVMs) is extended to least-squares SVM formulations that include LS-SVM classification, kernel ridge regression, and a particular form of regularized kernel Fisher discriminant to be asymptotically convergent.
Abstract
This article extends the well-known SMO algorithm of support vector machines (SVMs) to least-squares SVM formulations that include LS-SVM classification, kernel ridge regression, and a particular form of regularized kernel Fisher discriminant. The algorithm is shown to be asymptotically convergent. It is also extremely easy to implement. Computational experiments show that the algorithm is fast and scales efficiently (quadratically) as a function of the number of examples.
Keywords
Computer ScienceEngineering
ACM Transactions on Intelligent Systems and TechnologyLIBSVM
41,340 Citations2011Chih-Chung Chang, Chih‐Jen Lin
Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
Statistics and ComputingA tutorial on support vector regression
13,005 Citations2004Alex Smola, Bernhard Schölkopf
This tutorial gives an overview of the basic ideas underlying Support Vector (SV) machines for function estimation, and includes a summary of currently used algorithms for training SV machines, covering both the quadratic programming part and advanced methods for dealing with large datasets.
Neural Processing LettersLeast Squares Support Vector Machine Classifiers
9,375 Citations1999Johan A. K. Suykens, Joos Vandewalle
A least squares version for support vector machine (SVM) classifiers that follows from solving a set of linear equations, instead of quadratic programming for classical SVM's.
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
4,863 Citations1999John Platt
The output of a lassi(cid:12)er should be a alibrated posterior probability to enable post-pro essing and a method to train a kernel lassi with a logit link and a regularized maximum likelihood is proposed.
Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines
2,685 Citations1998John Platt
Fisher discriminant analysis with kernels
2,661 Citations2003Mika Sirén, Gunnar Rätsch +3 more
A non-linear classification technique based on Fisher's discriminant which allows the efficient computation of Fisher discriminant in feature space and large scale simulations demonstrate the competitiveness of this approach.
Neural ComputationImprovements to Platt's SMO Algorithm for SVM Classifier Design
1,817 Citations2001S. Sathiya Keerthi, Shirish Shevade +2 more
Using clues from the KKT conditions for the dual problem, two threshold parameters are employed to derive modifications of SMO that perform significantly faster than the original SMO on all benchmark data sets tried.
ePrints Soton (University of Southampton)Ridge Regression Learning Algorithm in Dual Variables
802 Citations1998Craig Saunders, Alex Gammerman +1 more
A regression estimation algorithm which is a combination of the dual version of Ridge Regression is applied to the ANOVA enhancement of the infinitenode splines and the use of kernel functions, as used in Support Vector methods is introduced.
Machine LearningBenchmarking Least Squares Support Vector Machine Classifiers
749 Citations2003Tony Van Gestel, Johan A. K. Suykens +6 more
Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances, consistently very good when compared to a variety of methods described in the literature.
Optimization methods & softwareLOQO:an interior point code for quadratic programming
472 Citations1999Robert J. Vanderbei
This paper describes a software package, called LOQO, which implements a primal-dual interior-point method for general nonlinear programming, and shows that the industry standard MPS format can be nicely formulated in such a way to provide quasidefiniteness.
Neural ComputationBayesian Framework for Least-Squares Support Vector Machine Classifiers, Gaussian Processes, and Kernel Fisher Discriminant Analysis
293 Citations2002Tony Van Gestel, Johan A. K. Suykens +4 more
The LS-SVM formulation has clear primal-dual interpretations, and without the bias term, one explicitly constructs a model that yields the same expressions as have been obtained with GPs for regression, which has advantages for deriving analytic expressions in a Bayesian evidence framework.
Least squares support vector machine classifiers: a large scale algorithm
221 Citations1999Johan A. K. Suykens, Luděk Lukáš +2 more
An iterative training algorithm for LS-SVM's which is based on a conjugate gradient method which enables solving large scale classification problems which is illustrated on a multi two-spiral benchmark problem.
A Mathematical Programming Approach to the Kernel Fisher Algorithm
161 Citations2000Sebastian Mika, Gunnar Rätsch +1 more
It is found that both, KFD and the proposed sparse KFD, can be understood in an unifying probabilistic context and connections to Support Vector Machines and Relevance Vector Machines are shown.
IEEE Transactions on Neural NetworksBayesian Support Vector Regression Using a Unified Loss Function
138 Citations2004Wei Chu, S.S. Keerthi +1 more
Experimental results on simulated and real-world data sets indicate that the approach works well even on large data sets, and has the advantages of Bayesian methods for model adaptation and error bars of its predictions.
IEEE Transactions on Neural NetworksAn Improved Conjugate Gradient Scheme to the Solution of Least Squares SVM
95 Citations2005Wei Chu, C.J. Ong +1 more
An improved method to the numerical solution of LS-SVM is proposed and it is shown that the problem can be solved using one reduced system of linear equations.
Alpha seeding for support vector machines
81 Citations2000Dennis DeCoste, Kiri L. Wagstaff
A key practical obstacle in applying support vector machines to many large-scale data mining tasks is that SVM's generally scale quadratically (or worse) in the number of examples or support vectors.