Feature/Model Selection by the Linear Programming SVM Combined with State-of-Art Classifiers: What Can We Learn About the Data
IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural NetworksPublished 1 August 2007
Erinija Pranckevičienė, Ray Somorjai, Muoi N. Tran
Citations13
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This study investigates a feature selection strategy based on LPSVM as the initial feature filter, combined with state-of-art classification rules, and applies to five real-life datasets of the agnostic learning vs. prior knowledge challenge of IJCNN2007 to better understand the robustness of L PSVM as a feature filter.
Abstract
Many real-world classification problems are represented by very sparse and high-dimensional data. The recent successes of a linear programming support vector machine (LPSVM) for feature selection motivated a deeper analysis of the method when applied to sparse, multivariate data. Due to the sparseness, the selection of a classification model is greatly influenced by the characteristics of that particular dataset. In this study, we investigate a feature selection strategy based on LPSVM as the initial feature filter, combined with state-of-art classification rules, and apply to five real-life datasets of the agnostic learning vs. prior knowledge challenge of IJCNN2007. Our goal is to better understand the robustness of LPSVM as a feature filter. Our analysis suggests that LPSVM can be a useful black box method for identification of the profile of the informative features in the data. If the data are complex and better separable by nonlinear methods, then feature pre-filtering by LPSVM enhances the data representation for other classifiers.
Keywords
Computer Science
Artificial IntelligenceWrappers for feature subset selection
8,925 Citations1997Ron Kohavi, George H. John
The wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain and compares the wrapper approach to induction without feature subset selection and to Relief, a filter approach tofeature subset selection.
Advances in industrial controlPattern Classification
7,884 Citations2000Evan L. Russell, Leo H. Chiang +1 more
American Mathematical MonthlyCombinatorial Optimization: Algorithms and Complexity.
6,028 Citations1984David Johnson, Christos H. Papadimitriou +1 more
Proceedings of the National Academy of SciencesSelection bias in gene extraction on the basis of microarray gene-expression data
1,468 Citations2002Christophe Ambroise, Geoffrey J. McLachlan
It is demonstrated that when correction is made for the selection bias, the cross-validated error is no longer zero for a subset of only a few genes.
BioinformaticsClass prediction and discovery using gene microarray andproteomics mass spectroscopy data: curses, caveats, cautions
332 Citations2003Ray Somorjai, B. Dolenko +1 more
This work shows for several publicly available microarray and proteomics datasets how the 'curse of dimensionality' and dataset sparsity influence classification outcomes, and suggests an approach to assess the relative quality of apparently equally good classifiers.
Computational Optimization and ApplicationsA Feature Selection Newton Method for Support Vector Machine Classification
284 Citations2004Glenn Fung, O. L. Mangasarian
The proposed stand-alone Newton method can handle classification problems in very high dimensional spaces, and generates a classifier that depends on very few input features, such as 7 out of the original 28,032.
IEEE Transactions on Pattern Analysis and Machine IntelligenceWhat size test set gives good error rate estimates?
152 Citations1998Isabelle Guyon, J. Makhoul +2 more
This work addresses the problem of determining what size test set guarantees statistically significant results in a character recognition task, as a function of the expected error rate, by providing a statistical analysis showing that if, for example, the expected character error rate is around 1 percent, then, with a test set of at least 10,000 statistically independent handwritten characters, that is guaranteed.
Simultaneous feature selection and classifier training via linear programming: a case study for face expression recognition
74 Citations2003Guodong Guo, Charles R. Dyer
The linear programming technique used in this paper, which is called feature selection via linear programming (FSLP), can determine the number of features and which features to use in the resulting classification function based on recent results in optimization.
Signal ProcessingSimultaneous classification and relevant feature identification in high-dimensional spaces: application to molecular profiling data
61 Citations2003Chiranjib Bhattacharyya, Leslie R. Grate +6 more
A role for the cellular microenvironment in influencing disease outcome and its importance in developing clinical decision support systems is suggested and LIKNON, a specific implementation of a statistical approach for creating a classifier and identifying a small number of relevant features simultaneously is described.
BioinformaticsAvoiding model selection bias in small-sample genomic datasets
56 Citations2006Daniel Berrar, Ian Bradbury +1 more
This study investigates the conceptual difficulties involved in comparative classification studies by means of a comparison of various classifiers in the context of multiclass microarray data and presents a statistical methodology that avoids bias in cross-validated model selection in the contexts of small-sample scenarios.
ACM SIGKDD Explorations NewsletterSupervised analysis when the number of candidate features (p) greatly exceeds the number of cases (n)
53 Citations2003Richard Simon
Some of key aspects of p>>n problems for identifying informative features and developing accurate classifiers are highlighted.
IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural NetworksAgnostic Learning vs. Prior Knowledge Challenge
35 Citations2007Isabelle Guyon, Amir Saffari +2 more
A challenge for IJCNN 2007, in which participants were allowed to compete in two tracks: the "prior knowledge" (PK) track, for which they had access to the original raw data representation and as much knowledge as possible about the data, and the "agnostic learning" (AL) track for which they were forced to use data pre-formatted as a table with dummy features.
Pattern Analysis and ApplicationsRegularisation of Linear Classifiers by Adding Redundant Features
15 Citations1999Marina Skurichina, Robert P. W. Duin
Noise injection by adding redundant features to the data is similar to other regularisation techniques, and helps to improve the generalisation error of this classifier for critical training sample sizes.
Class Separability in Spaces Reduced By Feature Selection
14 Citations2006Erinija Pranckevičienė, Tin Kam Ho +1 more
The study shows that geometrical complexities have good potentials for comparing different feature selection methods in aspects relevant to classification accuracy, yet independent of particular classifier choices.
Artificial Intelligence in MedicineIdentification of signatures in biomedical spectra using domain knowledge
12 Citations2005Erinija Pranckevičienė, Ray Somorjai +2 more
It is demonstrated that incorporating domain knowledge into feature selection methods helps identify interpretable features with predictive capability comparable to a state-of-the-art classifier.
Lecture notes in computer scienceControl of Sparseness for Feature Selection
1 Citations2004Erinija Pranckevičienė, Richard Baumgartner +2 more
By qualitatively analyzing the objective function and the constraints of the problem, this work shows why sparseness arises in a sparse solution.