Speeding Up Logistic Model Tree Induction

This work addresses the issue of overfitting of logistic model trees by using the AIC criterion instead of cross-validation to prevent overfitting, and a weight trimming heuristic is used which produces a significant speedup.

Abstract

Logistic Model Trees have been shown to be very accurate and compact classifiers [8]. Their greatest disadvantage is the computational complexity of inducing the logistic regression models in the tree. We address this issue by using the AIC criterion [1] instead of cross-validation to prevent overfitting these models. In addition, a weight trimming heuristic is used which produces a significant speedup. We compare the training time and accuracy of the new induction process with the original one on various datasets and show that the training time often decreases while the classification accuracy diminishes only slightly.