Classifiers in almost empty spaces
Published 11 November 2002
Robert P. W. Duin
Citations136
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It can be explained and illustrated how recognition systems based on support vector machines and subspace classifiers circumvent the curse of dimensionality, and even may find nonlinear decision boundaries for small training sets represented in Hilbert space.
Abstract
Recent developments in defining and training statistical classifiers make it possible to build reliable classifiers in very small sample size problems. Using these techniques advanced problems may be tackled, such as pixel based image recognition and dissimilarity based object classification. It can be explained and illustrated how recognition systems based on support vector machines and subspace classifiers circumvent the curse of dimensionality, and even may find nonlinear decision boundaries for small training sets represented in Hilbert space.
Keywords
Computer ScienceEngineering
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Data Mining and Knowledge DiscoveryA Tutorial on Support Vector Machines for Pattern Recognition
16,433 Citations1998Christopher J. C. Burges
There are several arguments which support the observed high accuracy of SVMs, which are reviewed and numerous examples and proofs of most of the key theorems are given.
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
Neural ComputationNonlinear Component Analysis as a Kernel Eigenvalue Problem
8,078 Citations1998Bernhard Schölkopf, Alexander J. Smola +1 more
A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.
IEEE Transactions on Electronic ComputersGeometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition
2,037 Citations1965Thomas M. Cover
It is shown that a family of surfaces having d degrees of freedom has a natural separating capacity of 2d pattern vectors, thus extending and unifying results of Winder and others on the pattern-separating capacity of hyperplanes.
IEEE Transactions on Pattern Analysis and Machine IntelligenceSmall sample size effects in statistical pattern recognition: recommendations for practitioners
1,366 Citations1991Šarūnas Raudys, Anil K. Jain
The effects of sample size on feature selection and error estimation for several types of classifiers are discussed and an emphasis is placed on giving practical advice to designers and users of statistical pattern recognition systems.
Handbook of statistics39 Dimensionality and sample size considerations in pattern recognition practice
556 Citations1982A. Jain, B. Chandrasekaran
It is shown that as the number of samples increases, not only does the designer have more confidence in the performance of the classifier, but also more measurements can be incorporated in the design of the classify without the fear of peaking in its performance.
Pattern Recognition LettersExpected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix
185 Citations1998Šarūnas Raudys, Robert P. W. Duin
An asymptotic formula for the expected (generalization) error of the Fisher classifier with the pseudo-inversion is derived which explains the peaking behaviour: with an increasing number of learning observations from one up to the number of features, the generalization error first decreases, and then starts to increase.
IEEE Transactions on Pattern Analysis and Machine IntelligenceRepresentation and recognition of handwritten digits using deformable templates
182 Citations1997Anil K. Jain, Douglas E. Zongker
The application of deformable templates to recognition of handprinted digits shows that there does exist a good low-dimensional representation space and methods to reduce the computational requirements, the primary limiting factor, are discussed.
Pattern Recognition LettersExperiments with a featureless approach to pattern recognition
69 Citations1997Robert P. W. Duin, Dick de Ridder +1 more
This paper discusses the possibility to construct classifiers entirely based on distances or similarities, without a relation with the feature space, as an alternative to automatic pattern recognition.
Perspectives in neural computingSupport objects for domain approximation
65 Citations1998Alexander Ypma, Robert P. W. Duin
A novel algorithm for extracting samples from a data set supporting the extremal points in the set, which could enable adaptation to novel (e.g. machine wear) data.
Classifiers for dissimilarity-based pattern recognition
54 Citations2002Elżbieta Pękalska, Robert P. W. Duin
A number of methods to tackle this type of classification problem are investigated: the feature- based (interpreting the distance representation as a feature space) and rank-based (considering the given relations) decision rules.
Pattern Recognition LettersRelational discriminant analysis
52 Citations1999Robert P. W. Duin, Elżbieta Pękalska +1 more
This paper shows that this subset of objects in the training data might be small and that its exact choice is of minor importance, and it is shown that linear or non-linear methods for feature extraction based on multi-dimensional scaling are not, or just hardly better than subsets.
IEEE Transactions on Neural NetworksGrowing subspace pattern recognition methods and their neural-network models
29 Citations1997M. Prakash, M. Narasimha Murty
This paper proposes two new algorithms, along with their neural-network implementations, to overcome certain limitations of the earlier LSMs and indicates that the proposed classifiers are comparable to classifiers like the multilayer perceptrons and the nearest-neighbor classifier in terms of classification accuracy.
Pattern Analysis and ApplicationsRegularisation of Linear Classifiers by Adding Redundant Features
15 Citations1999Marina Skurichina, Robert P. W. Duin
Noise injection by adding redundant features to the data is similar to other regularisation techniques, and helps to improve the generalisation error of this classifier for critical training sample sizes.
Czech Digital Mathematics Library (Institute of Mathematics CAS)Featureless pattern classification
9 Citations1998Robert P. W. Duin, Dick de Ridder +1 more
In this paper the possibilities are discussed for training statistical pattern recognizers based on a distance representation of the objects instead of a feature representation, which corresponds with determining classification functions in Hilbert space using an infinite feature set.
Neural network experiences between perceptrons and support vectors.
7 Citations1997Robert P. W. Duin, Dick de Ridder
A summary will be presented of the insights and fruits resulting from the neural network hype with an emphasis on the recently developed support vector machines.
Relational discriminant analysis and its large sample size problem
5 Citations2002Robert P. W. Duin
This work discusses and initially evaluates a solution that drastically decreases training times and memory demands based on a similarity matrix of the training set.