Distributed feature selection: An application to microarray data classification
Applied Soft ComputingPublished 7 February 2015
Verónica Bolón‐Canedo, Noelia Sánchez‐Maroño, Amparo Alonso‐Betanzos
Citations179
SJR quartileQ1
SJR score1.51
SNIP1.97
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The results on eight microarray datasets show that the execution time is considerably shortened whereas the performance is maintained or even improved compared to the standard algorithms applied to the non-partitioned datasets.
Abstract
Feature selection is often required as a preliminary step for many pattern recognition problems. However, most of the existing algorithms only work in a centralized fashion, i.e. using the whole dataset at once. In this research a new method for distributing the feature selection process is proposed. It distributes the data by features, i.e. according to a vertical distribution, and then performs a merging procedure which updates the feature subset according to improvements in the classification accuracy. The effectiveness of our proposal is tested on microarray data, which has brought a difficult challenge for researchers due to the high number of gene expression contained and the small samples size. The results on eight microarray datasets show that the execution time is considerably shortened whereas the performance is maintained or even improved compared to the standard algorithms applied to the non-partitioned datasets.
Keywords
Computer ScienceBiochemistry, Genetics and Molecular Biology
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
ACM SIGKDD Explorations NewsletterThe WEKA data mining software
17,849 Citations2009Mark Hall, Eibe Frank +4 more
This paper provides an introduction to the WEKA workbench, reviews the history of the project, and, in light of the recent 3.6 stable release, briefly discusses what has been added since the last stable version (Weka 3.4) released in 2003.
Machine LearningInduction of Decision Trees
14,815 Citations1986J. R. Quinlan
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, which is described in detail.
Machine LearningInduction of decision trees
12,383 Citations1986J. R. Quinlan
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, which is described in detail.
Journal of Machine Learning ResearchAn introduction to variable and feature selection
7,831 Citations2003GuyonIsabelle, ElisseeffAndré
An introduction to variable and feature selection
6,577 Citations2009Isabelle Guyon
BioinformaticsA review of feature selection techniques in bioinformatics
5,289 Citations2007Yvan Saeys, Iñaki Inza +1 more
A basic taxonomy of feature selection techniques is provided, providing their use, variety and potential in a number of both common as well as upcoming bioinformatics applications.
Machine LearningInstance-Based Learning Algorithms
4,117 Citations1991David W. Aha, Dennis Kibler +1 more
This paper describes how storage requirements can be significantly reduced with, at most, minor sacrifices in learning rate and classification accuracy and extends the nearest neighbor algorithm, which has large storage requirements.
Correlation-based Feature Selection for Machine Learning
3,498 Citations1998Mark Hall
This thesis addresses the problem of feature selection for machine learning through a correlation based approach with CFS (Correlation based Feature Selection), an algorithm that couples this evaluation formula with an appropriate correlation measure and a heuristic search strategy.
Elsevier eBooksA Practical Approach to Feature Selection
2,954 Citations1992Kenji Kira, Larry Rendell
Comparison with other feature selection algorithms shows Relief's advantages in terms of learning time and the accuracy of the learned concept, suggesting Relief's practicality.
Lecture notes in computer scienceEstimating attributes: Analysis and extensions of RELIEF
2,950 Citations1994Igor Kononenko
In the context of machine learning from examples this paper deals with the problem of estimating the quality of attributes with and without dependencies among them and is analysed and extended to deal with noisy, incomplete, and multi-class data sets.
Machine LearningInstance-based learning algorithms
2,915 Citations1991David W. Aha, Dennis Kibler +1 more
This paper describes how storage requirements can be significantly reduced with, at most, minor sacrifices in learning rate and classification accuracy and extends the nearest neighbor algorithm, which has large storage requirements.
Journal of Bioinformatics and Computational BiologyMINIMUM REDUNDANCY FEATURE SELECTION FROM MICROARRAY GENE EXPRESSION DATA
2,794 Citations2005Chris Ding, HANCHUAN PENG
An empirical study of the naive Bayes classifier
2,549 Citations2001Irina Rish
This work analyzes the impact of the distribution entropy on the classification error, showing that low-entropy feature distributions yield good performance of naive Bayes and demonstrates that naive Baye works well for certain nearlyfunctional feature dependencies.
Feature selection for high-dimensional data: a fast correlation-based filter solution
2,213 Citations2003Lei Yu, Huan Liu
A novel concept, predominant correlation, is introduced, and a fast filter method is proposed which can identify relevant features as well as redundancy among relevant features without pairwise correlation analysis.
Efficient Feature Selection via Analysis of Relevance and Redundancy
1,977 Citations2004Lei Yu, Huan Liu
It is shown that feature relevance alone is insufficient for efficient feature selection of high-dimensional data, and a new framework is introduced that decouples relevance analysis and redundancy analysis.
Toward optimal feature selection
1,444 Citations1996Daphne Koller, Mehran Sahami
An efficient algorithm for feature selection which computes an approximation to the optimal feature selection criterion is given, showing that the algorithm effectively handles datasets with a very large number of features.
IEEE Transactions on Knowledge and Data EngineeringBenchmarking attribute selection techniques for discrete class data mining
1,231 Citations2003Mark A. Hall, Geoffrey Holmes
A benchmark comparison of several attribute selection methods for supervised classification by cross-validating the attribute rankings with respect to a classification learner to find the best attributes.
Artificial IntelligenceConsistency-based search in feature selection
895 Citations2003Manoranjan Dash, Huan Liu
An empirical study is conducted to examine the pros and cons of these search methods, give some guidelines on choosing a search method, and compare the classifier error rates before and after feature selection.
Knowledge and Information SystemsA review of feature selection methods on synthetic data
826 Citations2012Verónica Bolón‐Canedo, Noelia Sánchez‐Maroño +1 more
Several synthetic datasets are employed for this purpose, aiming at reviewing the performance of feature selection methods in the presence of a crescent number or irrelevant features, noise in the data, redundancy and interaction between attributes, as well as a small ratio between number of samples and number of features.
Proceedings of the 19th ACM international conference on Information and knowledge management
620 Citations2010
This year CIKM has received a record high number of submissions, as can be seen from the following statistics: 1382 abstracts submitted, 945 full papers plus 38 demo papers submitted, and 126 papers accepted for presentation as full papers and an additional 165 accepted for short papers.
IEEE/ACM Transactions on Computational Biology and BioinformaticsA Survey on Filter Techniques for Feature Selection in Gene Expression Microarray Analysis
618 Citations2012Cosmin Lazar, Jonatan Taminau +8 more
This survey focuses on filter feature selection methods for informative feature discovery in gene expression microarray (GEM) analysis, which is also known as differentially expressed genes (DEGs) discovery, gene prioritization, or biomarker discovery, and presents them in a unified framework.
MPG.PuRe (Max Planck Society)Feature selection via dependence maximization
346 Citations2012Le Song, Alex Smola +3 more
This work introduces a framework for feature selection based on dependence maximization between the selected features and the labels of an estimation problem, using the Hilbert-Schmidt Independence Criterion, and shows that a number of existing feature selectors are special cases of this framework.
Feature selection for ensembles
304 Citations1999David W. Opitz
This paper presents an ensemble feature selection approach that is based on genetic algorithms and shows improved performance over the popular and powerful ensemble approaches of AdaBoost and Bagging and demonstrates the utility of ensemble features selection.
Information SciencesSimultaneous feature selection and classification using kernel-penalized support vector machines
255 Citations2010Sebastián Maldonado, Richard W. Weber +1 more
An embedded method that simultaneously selects relevant features during classifier construction by penalizing each feature's use in the dual formulation of support vector machines (SVM) called kernel-penalized SVM (KP-SVM).
Computational Statistics & Data AnalysisTaxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography
250 Citations2009Lior Rokach
An updated survey of ensemble methods in classification tasks is presented, while a new taxonomy for characterizing them is introduced, based on five dimensions: inducer, combiner, diversity, size, and members' dependency.
Redundancy based feature selection for microarray data
246 Citations2004Yu Lei, Huan Liu
The relationship between feature relevance and redundancy is studied and an efficient method that can effectively remove redundant genes is proposed that has been demonstrated through an empirical study using public microarray data sets.
International Joint Conference on Artificial IntelligenceSearching for interacting features
215 Citations2007Zheng Zhao, Huan Liu
IEEE/ACM Transactions on Computational Biology and BioinformaticsA Top-r Feature Selection Algorithm for Microarray Gene Expression Data
195 Citations2011Seiya Imoto, Satoru Miyano
The proposed algorithm first divides genes into subsets, the sizes of which are relatively small, then selects informative smaller subsets of genes from a subset and merges the chosen genes with another gene subset to update the gene subset.
Pattern RecognitionAn ensemble of filters and classifiers for microarray data classification
185 Citations2011Verónica Bolón‐Canedo, Noelia Sánchez‐Maroño +1 more
An ensemble of filters and classifiers is described to reduce the variability of the features selected by filters in different classification domains and its adequacy was demonstrated by employing 10 microarray data sets.
Applied Soft ComputingA novel hybrid feature selection method for microarray data analysis
176 Citations2009Chien-Pang Lee, Yungho Leu
This paper proposes a novel hybrid method for feature selection in microarray data analysis that uses a genetic algorithm with dynamic parameter setting (GADP) to generate a number of subsets of genes and to rank the genes according to their occurrence frequencies in the gene subsets.
Combining Nearest Neighbor Classifiers Through Multiple Feature Subsets
162 Citations1998Stephen D. Bay
MFS, a combining algorithm designed to improve the accuracy of the nearest neighbor (NN) classi er, is presented and it is shown that MFS is able to reduce both bias and variance components of error.
Toward parallel and distributed learning by meta-learning
151 Citations1993Philip K. Chan, Salvatore J. Stolfo
Several meta-learning strategies for integrating independently learned classifiers by the same learner in a parallel and distributed computing environment are outlined, particularly suited for massive amounts of data that main-memory-based learning algorithms cannot efficiently handle.
Pattern RecognitionAn unsupervised approach to feature discretization and selection
136 Citations2011Artur Ferreira, Mário A. T. Figueiredo
This paper proposes combined unsupervised feature discretization and feature selection techniques, suitable for medium and high-dimensional datasets, and shows the efficiency of the proposed techniques as well as improvements over previous related techniques.
Computers in Biology and MedicineA hybrid feature selection method for DNA microarray data
115 Citations2011Li‐Yeh Chuang, Cheng‐Huei Yang +4 more
In this paper, correlation-based feature selection and the Taguchi-genetic algorithm and the TGA method were combined into a hybrid method, and the K-nearest neighbor with the leave-one-out cross-validation method served as a classifier for eleven classification profiles to calculate the classification accuracy.
Evaluation of SMOTE for High-Dimensional Class-Imbalanced Microarray Data
99 Citations2012Rok Blagus, Lara Lusa
It is observed that SMOTE does not attenuate the bias towards the classification in the majority class for most classifiers, and it is less effective than random undersampling, so SMOTE for k-NN without variable selection should be used in practice.
NeurocomputingData classification using an ensemble of filters
88 Citations2014Verónica Bolón‐Canedo, Noelia Sánchez‐Maroño +1 more
The adequacy of using an ensemble of filters rather than a single filter was demonstrated on synthetic and real data, paving the way for its final application over a challenging scenario such as DNA microarray classification.
Pattern RecognitionMaximum weight and minimum redundancy: A novel framework for feature subset selection
77 Citations2012Jianzhong Wang, Lishan Wu +3 more
A novel filter framework is presented to select optimal feature subset based on a maximum weight and minimum redundancy (MWMR) criterion that can select the feature subset in which the features are most beneficial to the subsequent tasks while the redundancy among them is minimal.
Journal of Parallel and Distributed ComputingDistributed prediction from vertically partitioned data
42 Citations2007David B. Skillicorn, Sabine McConnell
This work shows how to extend attribute ensembles to data that is partitioned both horizontally and vertically, and reduces the amount of communication required, distributes computation and data access well, and allows each local site to keep its raw data private.
Lecture notes in computer scienceScalable, Distributed and Dynamic Mining of Association Rules
38 Citations2000V. S. Ananthanarayana, D.K. Subramanian +1 more
It is shown that construction of this tree and then generation of all large itemsets requires a single database scan where as the current algorithms need at least two database scans.
Knowledge and Information SystemsA local asynchronous distributed privacy preserving feature selection algorithm for large peer-to-peer networks
36 Citations2009Kamalika Das, Kanishka Bhaduri +1 more
A local distributed privacy preserving algorithm for feature selection in a large peer-to-peer environment in an asynchronous fashion with a low communication overhead where each peer can specify its own privacy constraints.
Privacy preserving feature selection for distributed data using virtual dimension
36 Citations2011Madhushri Banerjee, Sumit Chakravarty
A secure distributed protocol is proposed that will allow feature selection for multiple parties without revealing their own data and the experimental results with real life datasets presented in this paper will demonstrate the effectiveness of the proposed method.
Lecture notes in computer scienceStochastic Attribute Selection Committees
35 Citations1998Zijian Zheng, Geoffrey I. Webb
An empirical evaluation of a variant of this method, namely Sasc, in a representative collection of natural domains shows that the SASC method can significantly reduce the error rate of decision tree learning.
Building predictors from vertically distributed data
30 Citations2004Sabine McConnell, David B. Skillicorn
This paper presents a novel ensemble approach, in which data is partitioned by attributes, which is suitable for physically distributed data as well as privacy preserving data mining and can even produce an increase in classification accuracy compared to a centralized technique.
Lecture notes in computer scienceHigh Performance Computing — HiPC 2001
15 Citations2001Burkhard Monien, Viktor K. Prasanna +2 more
This paper investigates and analyzes the stability properties of heterogeneous networks, which use a combination of different universally stable queueing policies for packet routing, in the Adversarial Queueing model, and proves that the combination of SIS and LIS policies, LIS and NTS policies, and L IS and FTG policies leads to instability for specific networks and injection rates.
Lecture notes in computer scienceMassively Parallel Feature Selection: An Approach Based on Variance Preservation
10 Citations2012Zheng Zhao, James A. Cox +2 more