Bernoulli Mixture Models for Markov Blanket Filtering and Classification

The use of Bernoulli mixture models for Markov blanket flltering and classiflcation of binary data is presented, overcoming the short comings of their algorithm and increasing the e‐ciency of this algorithm considerably.

Abstract

This paper presents the use of Bernoulli mixture models for Markov blanket flltering and classiflcation of binary data. Bernoulli mixture models can be seen as a tool for partitioning an n-dimensional hypercube, identifying regions of high data density on the corners of the hypercube. Once Bernoulli mixture models are computed from a training dataset we use them for determining the Markov blanket of the target variable. An algorithm for Markov blanket flltering was proposed by Koller and Sahami (1996), which is a greedy search method for feature subset selection and it outputs an approximation to the optimal feature selection criterion. However, they use the entire training instances for computing the conditioning sets and have to limit the size of these sets for computational e‐ciency and avoiding data fragmentation. We have adapted their algorithm to use Bernoulli mixture models instead, hence, overcoming the short comings of their algorithm and increasing the e‐ciency of this algorithm considerably. Once a feature subset is identifled we perform classiflcation using these mixture models. We have applied this algorithm to the causality challenge datasets. Our prediction scores were ranked fourth on SIDO and our feature scores were ranked the best for test sets 1 and 2 of the same dataset.