A Meta-Learning Method to Select the Kernel Width in Support Vector Regression
Machine LearningPublished 13 February 2004Open access
Carlos Soares, Pavel Brazdil, Petr Kuba
Citations183
SJR quartileQ1
SJR score1.15
SNIP2.14
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A meta-learning methodology that can select settings with low error while providing significant savings in time is proposed and applied to set the width of the Gaussian kernel.
Abstract
The Support Vector Machine algorithm is sensitive to the choice of parameter settings. If these are not set correctly, the algorithm may have a substandard performance. Suggesting a good setting is thus an important problem. We propose a meta-learning methodology for this purpose and exploit information about the past performance of different settings. The methodology is applied to set the width of the Gaussian kernel. We carry out an extensive empirical evaluation, including comparisons with other methods (fixed default ranking; selection based on cross-validation and a heuristic method commonly used to set the width of the SVM kernel). We show that our methodology can select settings with low error while providing significant savings in time. Further work should be carried out to see how the methodology could be adapted to different parameter setting tasks.
Keywords
Computer Science
Data Mining and Knowledge DiscoveryA Tutorial on Support Vector Machines for Pattern Recognition
16,433 Citations1998Christopher J. C. Burges
There are several arguments which support the observed high accuracy of SVMs, which are reviewed and numerous examples and proofs of most of the key theorems are given.
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
Journal of Computational and Graphical StatisticsR: A Language for Data Analysis and Graphics
10,424 Citations1996Ross Ihaka, Robert Gentleman
The experience designing and implementing a statistical computing language that provides advantages in the areas of portability, computational efficiency, memory management, and scoping is discussed.
Cambridge University Press eBooksPattern Recognition and Neural Networks
6,468 Citations1996B. D. Ripley
IEEE Transactions on Neural NetworksAn introduction to kernel-based learning algorithms
3,490 Citations2001K. Müller, S. Mika +3 more
This paper provides an introduction to support vector machines, kernel Fisher discriminant analysis, and kernel principal component analysis, as examples for successful kernel-based learning methods.
Machine LearningChoosing Multiple Parameters for Support Vector Machines
2,182 Citations2002Olivier Chapelle, Vladimir Vapnik +2 more
The problem of automatically tuning multiple parameters for pattern recognition Support Vector Machines (SVMs) is considered by minimizing some estimates of the generalization error of SVMs using a gradient descent algorithm over the set of parameters.
ACM SIGIR ForumIR evaluation methods for retrieving highly relevant documents
1,466 Citations2017Kalervo Järvelin, Jaana Kekäläinen
The novel evaluation methods and the case demonstrate that non-dichotomous relevance assessments are applicable in IR experiments, may reveal interesting phenomena, and allow harder testing of IR methods.
IR evaluation methods for retrieving highly relevant documents
958 Citations2000Kalervo Järvelin, Jaana Kekäläinen
The novel evaluation methods and the case demonstrate that non-dichotomous relevance assessments are applicable in IR experiments, may reveal interesting phenomena, and allow harder testing of IR methods.
Evaluation of Item-Based Top- <i>N</i> Recommendation Algorithms
585 Citations2001George Karypis
The experimental evaluation on five different datasets show that the proposed item-based algorithms are up to 28 times faster than the traditional user-neighborhood based recommender systems and provide recommendations whose quality is up to 27% better.
PubMedUsing the Fisher kernel method to detect remote protein homologies.
414 Citations1999Tommi Jaakkola, Mark Diekhans +1 more
A new method, called the Fisher kernel method, for detecting remote protein homologies is introduced and shown to perform well in classifying protein domains by SCOP superfamily.
Machine LearningRanking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results
404 Citations2003Pavel Brazdil, Carlos Soares +1 more
A meta-learning method that uses a k-Nearest Neighbor algorithm to identify the datasets that are most similar to the one at hand and leads to significantly better rankings than the baseline ranking method.
Meta-Learning by Landmarking Various Learning Algorithms
302 Citations2000Bernhard Pfahringer, Hilan Bensusan +1 more
Experiments show that landmarking selects, with moderate but reasonable level of success, the best performing of a set of learning algorithms.
Elsevier eBooksAutomatic Parameter Selection by Minimizing Estimated Error
198 Citations1995Ron Kohavi, George H. John
A “wrapper” method that uses best-first search and crossvalidation to wrap around the basic induction algorithm to find the parameter settings that will result in optimal performance of a given learning algorithm using a particular dataset as training data.
Dynamically Adapting Kernels in Support Vector Machines
153 Citations1998Nello Cristianini, Colin Campbell +1 more
In this procedure model selection and learning are not separate, but kernels are dynamically adjusted during the learning process to find the kernel parameter which provides the best possible upper bound on the generalisation error.
Choosing Multiple Parameters for Support Vector Machines
79 Citations2000Chapelle, O., Vapnik V, Bousquet, O. +1 more
From Regularization Operators to Support Vector Kernels
66 Citations1997Alex Smola, Bernhard Schölkopf
It is proved that the Green's Functions associated with regularization operators are suitable Support Vector Kernels with equivalent regularization properties and a large number of Radial Basis Functions namely conditionally positive definite functions may be used as Support Vector kernels.
Exploiting Sampling and Meta-learning for Parameter Setting for Support Vector Machines
31 Citations2002Petr Kuba, Pavel Brazdil +2 more
The first approach uses small samples, while the second one exploits meta-learning and past results to obtain quite good results with significant savings in experimentation time.