Confident wrapper-type semi-supervised feature selection using an ensemble classifier

Published 1 August 2011

Yongkoo Han, Kisung Park, Young-Koo Lee

DOI10.1109/aimsec.2011.6010202

Citations27

TL;DR

The relationship between wrapper-type feature selection and the confidence of unlabeled data is analyzed and how this relationship can make the semisupervised feature selection framework faster and more accurate is explored.

Abstract

Feature selection is an important data preprocessing step in pattern recognition. Recently, a wrapper-type semi-supervised feature selection method, known as FW-SemiFS, was proposed to overcome the small labeled sample problem of supervised feature selection. FW-SemiFS does not consider the confidence of predicted unlabeled data, but rather evaluates the relevance of features according to their frequency. Such frequencies are obtained via iterative supervised sequential forward feature selection (SFFS). However, the large amount of computational time associated with iterative SFFS is detrimental to FW-SemiFS. Furthermore, this relevance evaluation method eliminates the primary advantage of wrapper-type feature selection: the ability to evaluate the discriminative power of a combination of features. In this paper, we propose a new wrapper-type semi-supervised feature selection framework that can select a more relevant feature subset using confident unlabeled data. The proposed framework, called ensemble-based semi-supervised feature selection (EN-SemiFS), employs an ensemble classifier that supports the estimation of the confidence of unlabeled data. We analyzed the relationship between wrapper-type feature selection and the confidence of unlabeled data and explored how this relationship can make the semisupervised feature selection framework faster and more accurate. The experimental results revealed that the proposed method can select a more relevant feature subset when compared to existing methods.

Keywords

Computer Science

Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm

49,657 Citations1977A. P. Dempster, N. M. Laird +1 more

Applied Physics Letters10.1162/153244303322753616

3,568 Citations2000

The contributions of this special issue cover a wide range of aspects of variable selection: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.

View PDF

Machine LearningText Classification from Labeled and Unlabeled Documents using EM

2,749 Citations2000Kamal Nigam, Andrew Kachites McCallum +2 more

This paper shows that the accuracy of learned text classifiers can be improved by augmenting a small number of labeled training documents with a large pool of unlabeled documents, and presents two extensions to the algorithm that improve classification accuracy under these conditions.

View PDF

IEEE Transactions on Knowledge and Data EngineeringToward integrating feature selection algorithms for classification and clustering

2,680 Citations2005Huan Liu, Lei Yu

With the categorizing framework, the efforts toward-building an integrated system for intelligent feature selection are continued, and an illustrative example is presented to show how existing feature selection algorithms can be integrated into a meta algorithm that can take advantage of individual algorithms.

Intelligent Data AnalysisFeature selection for classification

2,602 Citations1997Manoranjan Dash, Hua Liu

IEEE Transactions on Pattern Analysis and Machine IntelligenceFeature selection: evaluation, application, and small sample performance

2,167 Citations1997Anil K. Jain, Douglas E. Zongker

This work studies the problem of choosing an optimal feature set for land use classification based on SAR satellite images using four different texture models and shows that pooling features derived from different texture Models, followed by a feature selection results in a substantial improvement in the classification accuracy.

IEEE Transactions on Geoscience and Remote SensingThe effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon

572 Citations1994B.M. Shahshahani, D. A. Landgrebe

By using additional unlabeled samples that are available at no extra cost, the performance may be improved, and therefore the Hughes phenomenon can be mitigated and therefore more representative estimates can be obtained.

Semi-supervised Feature Selection via Spectral Analysis

300 Citations2007Zheng Zhao, Huan Liu

A semi-supervised feature selection algorithm based on spectral analysis that exploits both labeled and unlabeled data through a regularization framework, which provides an effective way to address the “small labeled-sample” problem.

Neural Information Processing SystemsA Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data

289 Citations1996David J. Miller, Hasan S. Uyar

A classifier structure and learning algorithm that make effective use of unlabelled data to improve performance and is a "mixture of experts" structure that is equivalent to the radial basis function (RBF) classifier, but unlike RBFs, is amenable to likelihood-based training.

Computational LinguisticsUnderstanding the Yarowsky Algorithm

132 Citations2004Steven Abney

A number of variants of the Yarowsky algorithm (though not the original algorithm itself) are shown to optimize either likelihood or a closely related objective function K.

View PDF