Probability estimation for multi-class classification using AdaBoost

A theoretical analysis of probability estimation model for multi-class classification by pairwise coupling is put forward and some verification experiments indicate that AdaBoost can be used for probability estimation.

Abstract

It is a general viewpoint that AdaBoost classifier has excellent performance on classification problems but could not produce good probability estimations. In this paper we put forward a theoretical analysis of probability estimation model and present some verification experiments, which indicate that AdaBoost can be used for probability estimation. With the theory, we suggest some useful measures for using AdaBoost algorithms properly. And then we deduce a probability estimation model for multi-class classification by pairwise coupling. Unlike previous approximate methods, we provide an analytical solution instead of a special iterative procedure. Moreover, a new problem that how to get a robust prediction with classifier scores is proposed. Experiments show that the traditional predict framework, which chooses one with the highest score from all classes as the prediction, is not always good while our model performs well.