LIBSVM
ACM Transactions on Intelligent Systems and TechnologyPublished 1 April 2011
Chih-Chung Chang, Chih‐Jen Lin
Citations41,340
SJR quartileQ1
SJR score1.36
SNIP1.84
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
Abstract
LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
Keywords
Computer Science
Machine LearningSupport-Vector Networks
33,035 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
A training algorithm for optimal margin classifiers
11,594 Citations1992Bernhard E. Boser, Isabelle Guyon +1 more
A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented, applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions.
Neural ComputationEstimating the Support of a High-Dimensional Distribution
5,946 Citations2001Bernhard Schölkopf, John Platt +3 more
The algorithm is a natural extension of the support vector algorithm to the case of unlabeled data by carrying out sequential optimization over pairs of input patterns and providing a theoretical analysis of the statistical performance of the algorithm.
The MIT Press eBooksFast Training of Support Vector Machines Using Sequential Minimal Optimization
5,462 Citations1998John Platt
A Practical Guide to Support Vector Classication
5,069 Citations2008Chih‐Wei Hsu, Chih-Chung Chang +1 more
A simple procedure is proposed, which usually gives reasonable results and is suitable for beginners who are not familiar with SVM.
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
4,863 Citations1999John Platt
The output of a lassi(cid:12)er should be a alibrated posterior probability to enable post-pro essing and a method to train a kernel lassi with a logit link and a regularized maximum likelihood is proposed.
Technical reportsMaking Large-Scale SVM Learning Practical
4,317 Citations2006Thorsten Joachims
This chapter presents algorithmic and computational results developed for SVM light V 2.0, which make large-scale SVM training more practical and give guidelines for the application of SVMs to large domains.
Neural ComputationNew Support Vector Algorithms
2,816 Citations2000Bernhard Schölkopf, Alex Smola +2 more
A new class of support vector algorithms for regression and classification that eliminates one of the other free parameters of the algorithm: the accuracy parameter in the regression case, and the regularization constant C in the classification case.
Training support vector machines: an application to face detection
2,496 Citations2002E. Osuna, Robert M. Freund +1 more
A decomposition algorithm that guarantees global optimality, and can be used to train SVM's over very large data sets is presented, and the feasibility of the approach on a face detection problem that involves a data set of 50,000 data points is demonstrated.
Neural ComputationImprovements to Platt's SMO Algorithm for SVM Classifier Design
1,817 Citations2001S. Sathiya Keerthi, Shirish Shevade +2 more
Using clues from the KKT conditions for the dual problem, two threshold parameters are employed to derive modifications of SMO that perform significantly faster than the original SMO on all benchmark data sets tried.
Large Margin DAGs for Multiclass Classification
1,644 Citations1999John Platt, Nello Cristianini +1 more
An algorithm, DAGSVM, is presented, which operates in a kernel-induced feature space and uses two-class maximal margin hyperplanes at each decision-node of the DDAG, which is substantially faster to train and evaluate than either the standard algorithm or Max Wins, while maintaining comparable accuracy to both of these algorithms.
The pyramid match kernel: discriminative classification with sets of image features
1,494 Citations2005Kristen Grauman, Trevor Darrell
A new fast kernel function is presented which maps unordered feature sets to multi-resolution histograms and computes a weighted histogram intersection in this space and is shown to be positive-definite, making it valid for use in learning algorithms whose optimal solutions are guaranteed only for Mercer kernels.
Journal of Machine Learning ResearchProbability Estimates for Multi-class Classification by Pairwise Coupling
1,479 Citations2004Tingfan Wu, Chih‐Jen Lin +1 more
Journal of Machine Learning ResearchWorking Set Selection Using Second Order Information for Training Support Vector Machines
1,413 Citations2005Rong-En Fan, Pai-Hsuen Chen +1 more
A new technique for working set selection in SMO-type decomposition methods that uses second order information to achieve fast convergence andoretical properties such as linear convergence are established.
Interpolation of Scattered Data: Distance Matrices and Conditionally Positive Definite Functions
1,307 Citations1984Charles A. Micchelli
Machine LearningA note on Platt’s probabilistic outputs for support vector machines
868 Citations2007Hsuan-Tien Lin, Chih‐Jen Lin +1 more
An improved algorithm that theoretically converges and avoids numerical difficulties is proposed for Platt’s probabilistic outputs for Support Vector Machines.
NeurocomputingSingle-layer learning revisited: a stepwise procedure for building and training a neural network
825 Citations1990S. Knerr, L. Personnaz +1 more
A stepwise procedure for building and training a neural network intended to perform classification tasks, based on single layer learning rules, is presented, which breaks up the classification task into subtasks of increasing complexity in order to make learning easier.
Natural Language EngineeringMaltParser: A language-independent system for data-driven dependency parsing
805 Citations2007Joakim Nivre, Johan Hall +6 more
Experimental evaluation confirms that MaltParser can achieve robust, efficient and accurate parsing for a wide range of languages without language-specific enhancements and with rather limited amounts of training data.
DSpace@MIT (Massachusetts Institute of Technology)Support Vector Machines: Training and Applications
704 Citations1997E. Osuna, Robert M. Freund +1 more
RSVM: Reduced Support Vector Machines
644 Citations2001Yuh‐Jye Lee, O. L. Mangasarian
Computational results indicate that test set correctness for the reduced support vector machine (RSVM), with a nonlinear separating surface that depends on a small randomly selected portion of the dataset, is better than that of a conventional support vectors machine (SVM) with aNonlinear surface that explicitly depends on the entire dataset, and much better than a conventional SVM using a small random sample of the data.
Efficient svm training using low-rank kernel representations
595 Citations2002Shai Fine, Katya Scheinberg
This work shows that for a low rank kernel matrix it is possible to design a better interior point method (IPM) in terms of storage requirements as well as computational complexity and derives an upper bound on the change in the objective function value based on the approximation error and the number of active constraints (support vectors).
A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods
591 Citations2005Hsuan-Tien Lin
This paper discusses non-PSD kernels through the viewpoint of separability, and shows that the sigmoid kernel matrix is conditionally positive definite (CPD) in certain parameters and thus are valid kernels there.
NeuroinformaticsPyMVPA: a Python Toolbox for Multivariate Pattern Analysis of fMRI Data
528 Citations2009Michael Hanke, Yaroslav O. Halchenko +4 more
A Python-based, cross-platform, and open-source software toolbox, called PyMVPA, for the application of classifier-based analysis techniques to fMRI datasets, which makes use of Python’s ability to access libraries written in a large variety of programming languages and computing environments to interface with the wealth of existing machine learning packages.
Neural ComputationTraining <i>v</i>-Support Vector Classifiers: Theory and Algorithms
503 Citations2001Chih-Chung Chang, Chih‐Jen Lin
Applied Stochastic Models in Business and IndustryA tutorial on <i>ν</i>‐support vector machines
475 Citations2005Pai‐Hsuen Chen, Chih‐Jen Lin +1 more
The methods comprise the steps of providing a plastic extrudable food mass; providing at least one color; mixing the food mass and color to form an organized complexly patterned food dough having an initial cross sectional area.
IEEE Transactions on Neural NetworksA Study on SMO-Type Decomposition Methods for Support Vector Machines
345 Citations2006Pai-Hsuen Chen, Rong-En Fan +1 more
The main results include a simple asymptotic convergence proof, a general explanation of the shrinking and caching techniques, and the linear convergence of the methods.
Neural ComputationTraining <i>v</i>-Support Vector Regression: Theory and Algorithms
315 Citations2002Chih-Chung Chang, Chih‐Jen Lin
Some issues that do not occur in the case of classification are discussed, including the possible range of epsilon and the scaling of target values.
Machine LearningConvergence of a Generalized SMO Algorithm for SVM Classifier Design
306 Citations2002S. Sathiya Keerthi, Éric Gilbert
Convergence of a generalized version of the modified SMO algorithms given by Keerthi et al. for SVM classifier design is proved and the results are extended to modifiedSMO algorithms for solving ν-SVM classifiers problems.
Machine LearningA Simple Decomposition Method for Support Vector Machines
300 Citations2002Chih‐Wei Hsu, Chih‐Jen Lin
This paper through the design of decomposition methods for bound-constrained SVM formulations it is demonstrated that the working set selection is not a trivial task and a simple selection is proposed which leads to faster convergences for difficult cases.
Journal of Machine Learning ResearchBuilding Support Vector Machines with Reduced Classifier Complexity
266 Citations2006S. Sathiya Keerthi, Olivier Chapelle +1 more
A primal method that decouples the idea of basis functions from the concept of support vectors and greedily finds a set of kernel basis functions of a specified maximum size to approximate the SVM primal cost function well.
IEEE Transactions on Neural NetworksOn the convergence of the decomposition method for support vector machines
249 Citations2001Chih‐Jen Lin
The asymptotic convergence of the algorithm used by the software SVM(light) and other later implementation is proved and the size of the working set can be any even number.
IEEE Transactions on Neural NetworksThe analysis of decomposition methods for support vector machines
163 Citations2000Chih‐Chung Chang, Chih‐Wei Hsu +1 more
This paper connects this method to projected gradient methods and provides theoretical proofs for a version of decomposition methods and shows that this convergence proof is valid for general decomposition methods if their working set selection meets a simple requirement.
ePrints Soton (University of Southampton)Support Vector Machine Reference Manual
143 Citations1998Craig Saunders, Weston Stitson Mo +2 more
The Support Vector Machine (SVM) program allows a user to carry out pattern recognition and regression estimation, using support vector techniques on some given data.
IEEE Transactions on Neural NetworksAsymptotic convergence of an SMO algorithm without any assumptions
138 Citations2002Chih‐Jen Lin
IEEE Transactions on Neural NetworksA formal analysis of stopping criteria of decomposition methods for support vector machines
85 Citations2002Chih‐Jen Lin
A result shows that, in final iterations of the decomposition method, only a particular set of variables are still being modified, which supports the use of the shrinking and caching techniques in some existing implementations.
Institutional Research Information System (Università degli Studi di Trento)Fast and Scalable Local Kernel Machines
80 Citations2010Nicola Segata, Enrico Blanzieri
Machine LearningPolynomial-Time Decomposition Algorithms for Support Vector Machines
79 Citations2003Don Hush, Clint Scovel
This paper provides a model algorithm for decomposition, and shows that when the slack multiplier C satisfies √1/2 ≤ C ≤ mL, then it takes no more than 4LC2m4/∈ iterations to drive the criterion to within ∈ of its optimum.
Neural Information Processing SystemsA geometric interpretation of ν-SVM classifiers
75 Citations1999David J. Crisp, Christopher J. C. Burges
It is shown that the recently proposed variant of the Support Vector machine (SVM) algorithm, known as ν-SVM, can be interpreted as a maximal separation between subsets of the convex hulls of the data, which are called soft convex Hulls.
Maximum-Gain Working Set Selection for SVMs
68 Citations2006Tobias Glasmachers, Christian Igel
The convergence to an optimal solution of a variant termed hybrid maximum-gain working set selection is proved, which is empirically compared to the prominent most violating pair selection and the latest algorithm using second order information.
Optimization methods & softwareOn the convergence of a modified version of SVM<i><sup>light</sup></i>algorithm
37 Citations2005Laura Palagi, Marco Sciandrone
This work proposes a decomposition method on the basis of a proximal point modification of the subproblem and the basis of a working set selection rule that includes, as a particular case, the one used by the SVM light algorithm.
Journal of Machine Learning ResearchWorking Set Selection Using Second Order Information for Training Support Vector Machines
36 Citations2005FanRong-En, ChenPai-Hsuen +1 more
Journal of Machine Learning ResearchMaximum-Gain Working Set Selection for SVMs
28 Citations2006GlasmachersTobias, IgelChristian
Neural ComputationA Note on the Decomposition Methods for Support Vector Regression
24 Citations2002Shuo-Peng Liao, Hsuan-Tien Lin +1 more
Journal of Machine Learning ResearchGeneral Polynomial Time Decomposition Algorithms
18 Citations2007Nikolas List, Hans Ulrich Simon
BioinformaticsBDVal: reproducible large-scale predictive model development and validation in high-throughput datasets
7 Citations2010Kevin C. Dorff, Nyasha Chambwe +2 more
Journal of Machine Learning ResearchFast and Scalable Local Kernel Machines
4 Citations2010SegataNicola, BlanzieriEnrico
A computationally efficient approach to local learning with kernel methods is presented, and the Fast Local Kernel Support Vector Machine (FaLK-SVM) is presented.