Accurate On-line Support Vector Regression
Neural ComputationPublished 25 September 2003
Junshui Ma, James Theiler, Simon Perkins
Citations403
SJR quartileQ1
SJR score0.83
SNIP1.45
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
An accurate on-line support vector regression (AOSVR) that efficiently updates a trained SVR function whenever a sample is added to or removed from the training set.
Abstract
Batch implementations of support vector regression (SVR) are inefficient when used in an on-line setting because they must be retrained from scratch every time the training set is modified. Following an incremental support vector classification algorithm introduced by Cauwenberghs and Poggio (2001), we have developed an accurate on-line support vector regression (AOSVR) that efficiently updates a trained SVR function whenever a sample is added to or removed from the training set. The updated SVR function is identical to that produced by a batch algorithm. Applications of AOSVR in both on-line and cross-validation scenarios are presented. In both scenarios, numerical experiments indicate that AOSVR is faster than batch SVR algorithms with both cold and warm start.
Keywords
Computer ScienceEngineering
ACM Transactions on Intelligent Systems and TechnologyLIBSVM
41,340 Citations2011Chih-Chung Chang, Chih‐Jen Lin
Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Statistics and ComputingA tutorial on support vector regression
13,005 Citations2004Alex Smola, Bernhard Schölkopf
This tutorial gives an overview of the basic ideas underlying Support Vector (SV) machines for function estimation, and includes a summary of currently used algorithms for training SV machines, covering both the quadratic programming part and advanced methods for dealing with large datasets.
ScienceOscillation and Chaos in Physiological Control Systems
4,109 Citations1977Michael C. Mackey, Leon Glass
First-order nonlinear differential-delay equations describing physiological control systems displaying a broad diversity of dynamical behavior including limit cycle oscillations, with a variety of wave forms, and apparently aperiodic or "chaotic" solutions are studied.
OmegaApplication of support vector machines in financial time series forecasting
1,242 Citations2001Francis E. H. Tay, Lijuan Cao
Analysis of the experimental results proved that it is advantageous to apply SVMs to forecast !
Incremental and Decremental Support Vector Machine Learning
1,158 Citations2000Gert Cauwenberghs, Tomaso Poggio
An on-line recursive algorithm for training support vector machines, one vector at a time, is presented and interpretation of decremental unlearning in feature space sheds light on the relationship between generalization and geometry of the data.
IEEE Transactions on Signal ProcessingOnline Learning with Kernels
1,019 Citations2004Jyrki Kivinen, Alex Smola +1 more
This paper considers online learning in a reproducing kernel Hilbert space, and allows the exploitation of the kernel trick in an online setting, and examines the value of large margins for classification in the online setting with a drifting target.
Lecture notes in computer sciencePredicting time series with support vector machines
977 Citations1997K. Müller, Alexander J. Smola +4 more
Two different cost functions for Support Vectors are made use: training with an e insensitive loss and Huber's robust loss function and how to choose the regularization parameters in these models are discussed.
International Journal of ForecastingOut-of-sample tests of forecasting accuracy: an analysis and review
803 Citations2000Leonard J. Tashman
The structure of out-of-sample tests is explained, guidelines for implementing these tests are provided, and the adequacy of out of-offer tests in forecasting software is evaluated.
Neural ComputationBounds on Error Expectation for Support Vector Machines
627 Citations2000Vladimir Vapnik, Olivier Chapelle
It is proved that the value of the span is always smaller (and can be much smaller) than the diameter of the smallest sphere containing the support vectors, used in previous bounds.
Incremental learning with support vector machines
530 Citations2002Stefan Rüping
An approach for incremental learning with support vector machines is presented, that improves the existing approach of Syed et al. (1999), and an insight into the interpretability of support vectors is given.
Optimization methods & softwareLOQO:an interior point code for quadratic programming
472 Citations1999Robert J. Vanderbei
This paper describes a software package, called LOQO, which implements a primal-dual interior-point method for general nonlinear programming, and shows that the industry standard MPS format can be nicely formulated in such a way to provide quasidefiniteness.
Journal of Atmospheric and Solar-Terrestrial PhysicsRelativistic electron dynamics in the inner magnetosphere — a review
451 Citations2002R. H. W. Friedel, G. D. Reeves +1 more
Technische Universität Dortmund Eldorado (Technische Universität Dortmund)Estimating the generalization performance of a SVM efficiently
328 Citations1999Thorsten Joachims
Without any computation-intensive resampling, the new estimators developed here are computationally much more e cient than cross-validation or bootstrapping and address the special performancemeasures needed for evaluating text classi ers.
Neural ComputationTraining <i>v</i>-Support Vector Regression: Theory and Algorithms
315 Citations2002Chih-Chung Chang, Chih‐Jen Lin
Some issues that do not occur in the case of classification are discussed, including the possible range of epsilon and the scaling of target values.
Machine LearningThe Relaxed Online Maximum Margin Algorithm
230 Citations2002Yi Li, Philip M. Long
This work describes a new incremental algorithm for training linear threshold functions: the Relaxed Online Maximum Margin Algorithm, or ROMMA, and proves a mistake bound for ROMMA that is the same as that proved for the perceptron algorithm.
A New Approximate Maximal Margin Classification Algorithm
228 Citations2000Claudio Gentile
SIAM Journal on OptimizationWarm-Start Strategies in Interior-Point Methods for Linear Programming
153 Citations2002E. Alper Yıldırım, Stephen J. Wright
Worst-case estimates of the number of iterations required to converge to a solution of the perturbed instance from the warm-start points are obtained, showing that these estimates depend on the size ofThe perturbation and on the conditioning and other properties of the problem instances.
Sparse Representation for Gaussian Process Models
128 Citations2000Lehel Csató, Manfred Opper
An approach for a sparse representation for Gaussian Process models in order to overcome the limitations of GPs caused by large data sets is developed based on a combination of a Bayesian online algorithm together with a sequential construction of a relevant subsample of the data which fully specifies the prediction of the model.
Lecture notes in computer scienceIncremental Support Vector Machine Learning: A Local Approach
105 Citations2001Liva Ralaivola, Florence d’Alché–Buc
A new on-line algorithm for learning a SVM based on Radial Basis Function Kernel: Local Incremental Learning of SVM or LISVM, which exploits the "locality" of RBF kernels to update current machine by only considering a subset of support candidates in the neighbourhood of the input.
Mathematical ProgrammingWarm start of the primal-dual method applied in the cutting-plane scheme
94 Citations1998Jacek Gondzio
A practical warm-start procedure is described for the infeasible primal-dual interior-point method (IPM) employed to solve the restricted master problem within the cutting-plane method.
SIAM Journal on OptimizationReoptimization With the Primal-Dual Interior Point Method
62 Citations2002Jacek Gondzio, Andreas Grothey
Reoptimization techniques for an interior point method applied to solving a sequence of linear programming problems are discussed and numerical results with OOPS, a new object-oriented parallel solver, demonstrate the efficiency of the approach.
Lecture notes in computer scienceLearning Additive Models Online with Fast Evaluating Kernels
56 Citations2001Mark Herbster
Three new techniques are developed to build on the recent advances in online learning with kernels and show that an exponential speed-up in prediction time per trial is possible for such algorithms as the Kernel-Adatron, Kernel-Perceptron, and ROMMA for specific additive models.
On-line support vector machines for function approximation
53 Citations2002Mario Martín Muñoz
The method is an extension of the method developed by (Cauwenberghs & Poggio, 2000) for building incremental support vector machines for classification and opens the application of SVM regression to areas such as on-line prediction of temporal series or generalization of value functions in reinforcement learning.
From Margin to Sparsity
47 Citations2000Thore Graepel, Ralf Herbrich +1 more
An improvement of Novikoff's perceptron convergence theorem is presented, reinterpreting this mistake bound as a margin dependent sparsity guarantee to give a PAC-style generalisation error bound for the classifier learned by the perceptron learning algorithm.
ePrints Soton (University of Southampton)Constructing Approximations to the Efficient Set of Convex Quadratic Multiobjective Problems
24 Citations2002Jörg Fliege, Andree Heseler
A new efficient method for approximating the solution set of a convex quadratic multiobjective programming problem is proposed, based on a warm-start interior point algorithm for which complexity results are derived, thereby extending previous results by Yildirim & Wright.
Support vector pursuit learning
3 Citations2005Yangguang Liu, Qinming He +1 more
The proposed SVPL uses an incremental updating model similar to standard SVM to update the trained SVM parameters and provides the same learning performance as that obtained by batch learning, but is faster than other methods.