Combining feature ranking algorithms through rank aggregation
Published 1 June 2012Open access
Ronaldo C. Prati
Citations71
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A general framework for ensemble feature ranking is proposed, alongside four instantiations of this framework using different ranking aggregation methods, and one of the proposed methods was able to achieve results statistically significantly better than the others.
Abstract
The problem of combining multiple feature rankings into a more robust ranking is investigated. A general framework for ensemble feature ranking is proposed, alongside four instan-tiations of this framework using different ranking aggregation methods. An empirical evaluation using 39 UCI datasets, three different learning algorithms and three different performance measures enable us to reach a compelling conclusion: ensemble feature ranking do improve the quality of feature rankings. Furthermore, one of the proposed methods was able to achieve results statistically significantly better than the others.
Keywords
Computer Science
Machine LearningRandom Forests
126,192 Citations2001Leo Breiman
Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the forest, and are also applicable to regression.
Elsevier eBooksData Mining: Practical Machine Learning Tools and Techniques
25,718 Citations2011Ian H. Witten, Eibe Frank
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
Statistical Comparisons of Classifiers over Multiple Data Sets
11,198 Citations2006Janez Demšar
A set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers is recommended: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparisons of more classifiers over multiple data sets.
Machine LearningGene Selection for Cancer Classification using Support Vector Machines
9,839 Citations2002Isabelle Guyon, Jason Weston +2 more
This paper proposes a new method of gene selection utilizing Support Vector Machine methods based on Recursive Feature Elimination (RFE), and demonstrates experimentally that the genes selected yield better classification performance and are biologically relevant to cancer.
Lecture notes in computer scienceEnsemble Methods in Machine Learning
7,862 Citations2000Thomas G. Dietterich
Programs for Machine Learning
5,794 Citations1994Steven L. Salzberg, Alberto M. Segre
In his new book, C4.5: Programs for Machine Learning, Quinlan has put together a definitive, much needed description of his complete system, including the latest developments, which will be a welcome addition to the library of many researchers and students.
Applied Physics Letters10.1162/153244303322753616
3,568 Citations2000
The contributions of this special issue cover a wide range of aspects of variable selection: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.
Elsevier eBooksA Practical Approach to Feature Selection
2,954 Citations1992Kenji Kira, Larry Rendell
Comparison with other feature selection algorithms shows Relief's advantages in terms of learning time and the accuracy of the learned concept, suggesting Relief's practicality.
Lecture notes in computer scienceEstimating attributes: Analysis and extensions of RELIEF
2,950 Citations1994Igor Kononenko
In the context of machine learning from examples this paper deals with the problem of estimating the quality of attributes with and without dependencies among them and is analysed and extended to deal with noisy, incomplete, and multi-class data sets.
Machine LearningA Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
2,264 Citations2001David J. Hand, Robert John Till
This work extends the definition of the area under the ROC curve to the case of more than two classes by averaging pairwise comparisons and proposes an alternative definition of proportion correct based on pairwise comparison of classes for a simple artificial case.
Rank aggregation methods for the Web
1,798 Citations2001Cynthia Dwork, Ravi Kumar +2 more
A set of techniques for the rank aggregation problem is developed and compared to that of well-known methods, to design rank aggregation techniques that can be used to combat spam in Web searches.
Lecture notes in computer scienceRobust Feature Selection Using Ensemble Feature Selection Techniques
651 Citations2008Yvan Saeys, Thomas Abeel +1 more
It is shown that ensemble feature selection techniques show great promise for high-dimensional domains with small sample sizes, and provide more robust feature subsets than a single feature selection technique.
BioinformaticsRobust biomarker identification for cancer diagnosis with ensemble feature selection methods
584 Citations2009Thomas Abeel, Thibault Helleputte +3 more
It is shown that the robustness of SVMs for biomarker discovery can be substantially increased by using ensemble feature selection techniques, while at the same time improving upon classification performances.
IEEE Transactions on Pattern Analysis and Machine IntelligenceA Comparison of Decision Tree Ensemble Creation Techniques
372 Citations2006Robert E. Banfield, Lawrence Hall +2 more
An algorithm is introduced that decides when a sufficient number of classifiers has been created for an ensemble, and is shown to result in an accurate ensemble for those methods that incorporate bagging into the construction of the ensemble.
Conference on Artificial Intelligence for ApplicationsA stability index for feature selection
334 Citations2007Ludmila I. Kuncheva
A stability index is proposed here based on cardinality of the intersection and a correction for chance, and the experimental results indicate that the index can be useful for selecting the final feature subset.
Feature Selection with Ensembles, Artificial Variables, and Redundancy Elimination
278 Citations2009Eugene Tuv, Alexander Borisov +2 more
Social Choice and WelfareA new monotonic, clone-independent, reversal symmetric, and condorcet-consistent single-winner election method
265 Citations2010Markus Schulze
This article presents an O(C^3) algorithm to calculate the winner, where C is the number of alternatives and satisfies, e.g., resolvability, Condorcet, Pareto, reversal symmetry, monotonicity, and independence of clones.
Pattern Recognition LettersFeature selection in the independent component subspace for face recognition
102 Citations2004Hazım Kemal Ekenel, Bülent Sankur
Overall, face recognition benefits from the feature selection of ICA or PCA components and from the combination of I CA and PCA feature pools.
Pattern Recognition LettersConsensus unsupervised feature ranking from multiple views
80 Citations2007Yi Hong, Sam Kwong +2 more
The proposed FRMV method firstly obtains multiple rankings of all features from different views of the same data set and then aggregates all the obtained feature rankings into a single consensus one.
Lecture notes in computer scienceEnsemble Feature Ranking
65 Citations2004Kees H. de Jong, Jérémie Mary +3 more
An ensemble approach for Feature Ranking is proposed, aggregating feature rankings extracted along independent runs of an evolutionary learning algorithm named ROGER, inspired from the complexity framework proposed in the Constraint Satisfaction domain.
Pattern Recognition LettersSVM-based feature selection of latent semantic features
40 Citations2004Keisuke Shima, Masaru Todoriki +1 more
This work proposes to apply feature ordering method based on support vector machines in order to select LSI-features that are suited for classification and suggests that the method improves classification performance with considerably more compact representation.
Impact studies and sensitivity analysis in medical data mining with ROC-based genetic learning
14 Citations2004Michèle Sébag, Jérôme Azé +1 more
It is shown how a genetic algorithm-based optimization of the AUC criterion can be exploited for impact studies and sensitivity analysis on the Atherosclerosis Identification problem.
INTELIGENCIA ARTIFICIALA Simple Evaluation Model for Feature Subset Selection Algorithms.
7 Citations2006Huei Lee, Maria Carolina Monard +3 more
A simple model is proposed to evaluate FSS algorithms which considers two properties, namely the predictive performance of the classifier induced using the subset of features selected by different FSS algorithms, as well as the reduction in the number of features.
Lecture notes in computer scienceA Novel Ensemble of Distance Measures for Feature Evaluation: Application to Sonar Imagery
4 Citations2011Richard Harrison, Roger Birchall +2 more
A novel ensemble of probabilistic distance measures is proposed, as an objective function for evaluating features of qualitative sidescan sonar imagery, and a strong correlation is found between ensemble scores and classification accuracies.
El Servicio de Difusión de la Creación Intelectual (National University of La Plata)A hybrid wrapper/filter approach for feature subset selection
1 Citations2008Prati, Ronaldo C., Batista, Gustavo E. A. P. A. +1 more