Building Ensembles of Classifiers for Loss Minimization

Keywords: supervised learning, ensembles of classifiers, loss, misclassification costs

Abstract: One of the most active areas of research in supervised learning has been the study of methods for constructing good ensembles of classifiers (a set of classifiers whose individual decisions are combined to increase overall accuracy of classifying new examples).
In many applications classifiers are required to minimize an asymmetric loss function rather than the raw misclassification rate. In this paper, we study approaches to modifying existing methods for constructing ensembles, to incorporate arbitrary loss functions.
We compare our new algorithms with ensemble learning methods like boosting and bagging in which only the weak learner has been modified to incorporate the loss function, and with single hypotheses built by the weak learner. We also present a measure for estimating the complexity of the loss function.
In our experiments we used decision trees as weak hypotheses and evaluated our algorithms on multi-class data sets from the UC Irvine ML Repository.