A hybrid filter/wrapper approach of feature selection using information theory

This work begins with a filter model, exploiting the geometrical information contained in the minimum spanning tree (MST) built on the learning set, which leads to a feature selection algorithm belonging to a new category of hybrid models ( filter-wrapper).

Abstract

We focus on a hybrid approach of feature selection. We begin our analysis with a filter model, exploiting the geometrical information contained in the minimum spanning tree (MST) built on the learning set. This model exploits a statistical test of relative certainty gain, used in a forward selection algorithm. In the second part of the paper, we show that the MST can be replaced by the 1 nearest-neighbor graph without challenging the statistical framework. This leads to a feature selection algorithm belonging to a new category of hybrid models (filter-wrapper). Experimental results on readily available synthetic and natural domains are presented and discussed.