Extending naïve Bayes classifiers using long itemsets

It is shown that selecting and combining many and long itemsets providing new evidence is an effective strategy for computing the class probabilities and the proposed classification technique is called Large Bayes as it happens to reduce to Naive Bayes classifier when all itemsets selected are of size one only.

Abstract

Itemsets provide local descriptions of the data. This work proposes to use itemsets as basic means for classification purposes too. To enable this, the concept of class support SUpi of an itemset is introduced, i.e., how many times an itemset occurs when a specific class Ci is present. Class supports of frequent itemsets are computed in the training phase. Upon arrival of a new case to be classified, some of the generated itemsets are selected and their class supports SUpi are used to compute the probability that the case belongs to class cc The result is the class ci with highest such probability. We show that selecting and combining many and long itemsets providing new evidence (interesting) is an effective strategy for computing the class probabilities. The proposed classification technique is called Large Bayes as it happens to reduce to Naive Bayes classifier when all itemsets selected are of size one only. Experimental results on a large number of benchmark data sets show that Large Bayes consistently outperforms the widely used Ndive Bayes classifier. In many cases, Large Bayes is also superior to other state of the art classification methods such as C4.5, CBA (a recently proposed rule classifier built from association rules), and TAN (a Bayesian network extension of Ndive Bayes).