A Kernel Approach for Learning from almost Orthogonal Patterns
Lecture notes in computer sciencePublished 1 January 2002
Bernhard Schölkopf, Jason Weston, Eleazar Eskin, Christina S. Leslie, William Stafford Noble
Citations69
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
In kernel methods, all the information about the training data is contained in the Gram matrix. If this matrix has large diagonal values, which arises for many types of kernels, then kernel methods do not perform well. We propose and test several methods for dealing with this problem by reducing the dynamic range of the matrix while preserving the positive definiteness of the Hessian of the quadratic programming problem that one has to solve when training a Support Vector Machine.
Keywords
Computer Science
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
Machine LearningSupport-Vector Networks
33,035 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
A training algorithm for optimal margin classifiers
11,594 Citations1992Bernhard E. Boser, Isabelle Guyon +1 more
A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented, applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions.
NatureDistinct types of diffuse large B-cell lymphoma identified by gene expression profiling
9,951 Citations2000Ash A. Alizadeh, Michael B. Eisen +29 more
It is shown that there is diversity in gene expression among the tumours of DLBCL patients, apparently reflecting the variation in tumour proliferation rate, host response and differentiation state of the tumour.
Machine LearningGene Selection for Cancer Classification using Support Vector Machines
9,839 Citations2002Isabelle Guyon, Jason Weston +2 more
This paper proposes a new method of gene selection utilizing Support Vector Machine methods based on Recursive Feature Elimination (RFE), and demonstrates experimentally that the genes selected yield better classification performance and are biologically relevant to cancer.
Journal of Molecular BiologySCOP: A structural classification of proteins database for the investigation of sequences and structures
6,336 Citations1995Alexey G. Murzin, Steven E. Brenner +2 more
This database provides a detailed and comprehensive description of the structural and evolutionary relationships of the proteins of known structure and provides for each entry links to co-ordinates, images of the structure, interactive viewers, sequence data and literature references.
Proceedings of the National Academy of SciencesBroad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays
4,210 Citations1999Uri Alon, Naama Barkai +5 more
A two-way clustering method is reported for analyzing a data set consisting of the expression patterns of different cell types, revealing broad coherent patterns that suggest a high degree of organization underlying gene expression in these tissues.
Proceedings of the National Academy of SciencesKnowledge-based analysis of microarray gene expression data by using support vector machines
2,315 Citations2000Michael P. Brown, William Noble Grundy +6 more
A method of functionally classifying genes by using gene expression data from DNA microarray hybridization experiments, based on the theory of support vector machines (SVMs), to predict functional roles for uncharacterized yeast ORFs based on their expression data is introduced.
Information science and statisticsEstimation of Dependences Based on Empirical Data
2,238 Citations2006Vladimir Vapnik
Exploiting Generative Models in Discriminative Classifiers
1,285 Citations1998Tommi Jaakkola, David Haussler
A natural way of achieving this combination by deriving kernel functions for use in discriminative methods such as support vector machines from generative probability models is developed.
Graduate texts in mathematicsHarmonic Analysis on Semigroups
1,125 Citations1984Christian Berg, Jens Peter Christensen +1 more
THE SPECTRUM KERNEL: A STRING KERNEL FOR SVM PROTEIN CLASSIFICATION
944 Citations2001Christina S. Leslie, Eleazar Eskin +1 more
A new sequence-similarity kernel, the spectrum kernel, is introduced for use with support vector machines (SVMs) in a discriminative approach to the protein classification problem and performs well in comparison with state-of-the-art methods for homology detection.
MPG.PuRe (Max Planck Society)Use of the zero norm with linear models and kernel methods
700 Citations2003Jason Weston, André Elisseeff +2 more
A simple but practical method is derived for implementing a simple modification of vanilla SVM via an iterative multiplicative rescaling of the training data for minimizing the zero-norm of the parameters of linear models in learning.
Applied Physics Letters10.1162/153244302760200687
534 Citations2000
An approximation technique is introduced that is shown to deliver good approximations efficiently for large datasets and is shown to deliver good approximations efficiently for large datasets.
Journal of Computational BiologyA Discriminative Framework for Detecting Remote Protein Homologies
494 Citations2000Tommi Jaakkola, Mark Diekhans +1 more
A new method for detecting remote protein homologies is introduced and shown to perform well in classifying protein domains by SCOP superfamily using a new kernel function derived from a generative statistical model for a protein family, in this case a hidden Markov model.
The MIT Press eBooksDynamic Alignment Kernels
242 Citations2000Chris Watkins
This paper shows that the scores produced by certain dynamic align ment algorithms for sequences are in fact valid kernel functions, proved by expressing the alignment scores explicitly as dot products.
Combining pairwise sequence similarity and support vector machines for remote protein homology detection
145 Citations2002Li Liao, William Stafford Noble
The current work presents an alternative method for SVM-based protein classification that uses a pairwise sequence similarity algorithm such as Smith-Waterman in place of the HMM in the S VM-Fisher method, and yields significantly better remote protein homology detection.
Neural ComputationA New Discriminative Kernel from Probabilistic Models
129 Citations2002Koji Tsuda, Motoaki Kawanabe +3 more
This work proposes a new discriminative TOP kernel derived from tangent vectors of posterior log-odds and develops a theoretical framework on feature extractors from probabilistic models and uses it for analyzing the TOP kernel.
BioinformaticsFeature selection and transduction forprediction of molecular bioactivity for drug design
101 Citations2003Jason Weston, Fernando Pérez‐Cruz +4 more
Two methods for prediction of molecular bioactivity for drug design are introduced and shown to perform well in a data set previously studied as part of the KDD Cup 2001, characterized by very few positive examples, a very large number of features and rather different distributions between training and test data.
Annals of the Institute of Statistical MathematicsDealing with large diagonals in kernel matrices
39 Citations2003Jason Weston, Bernhard Schölkopf +3 more
Several methods are proposed and test for reducing the dynamic range of the matrix while preserving the positive definiteness of the Hessian of the quadratic programming problem that one has to solve when training a Support Vector Machine, which is a common kernel approach for pattern recognition.