Learning Multi-label Alternating Decision Trees from Texts and Data

Lecture notes in computer sciencePublished 1 January 2003Open access

Francesco de Comité, Rémi Gilleron, Marc Tommasi

Citations177

TL;DR

A supervised learning algorithm that extends Schapire and Singer's Adaboost and produces sets of rules that can be viewed as trees like Alternating Decision Trees (invented by Freund and Mason).

Abstract

Multi-label decision procedures are the target of the supervised learning algorithm we propose in this paper. Multi-label decision procedures map examples to a finite set of labels. Our learning algorithm extends Schapire and Singer's Adaboost.MH and produces sets of rules that can be viewed as trees like Alternating Decision Trees (invented by Freund and Mason). Experiments show that we take advantage of both performance and readability using boosting techniques as well as tree representations of large set of rules. Moreover, a key feature of our algorithm is the ability to handle heterogenous input data: discrete and continuous values and text data.

Keywords

Computer Science

C4.5: Programs for Machine Learning

23,665 Citations1992J. R. Quinlan

A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.

Journal of Computer and System SciencesA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting

20,320 Citations1997Yoav Freund, Robert E. Schapire

Experiments with a new boosting algorithm

7,585 Citations1996Yoav Freund, Robert E. Schapire

Journal of Artificial Intelligence ResearchSolving Multiclass Learning Problems via Error-Correcting Output Codes

2,718 Citations1995Tom Dietterich, Ghulum Bakiri

It is demonstrated that error-correcting output codes provide a general-purpose method for improving the performance of inductive learning programs on multiclass problems.

View PDF

Improved boosting algorithms using confidence-rated predictions

2,568 Citations1998Robert E. Schapire, Yoram Singer

Machine LearningBoosTexter: A Boosting-based System for Text Categorization

2,194 Citations2000Robert E. Schapire, Yoram Singer

This work describes in detail an implementation, called BoosTexter, of the new boosting algorithms for text categorization tasks, and presents results comparing the performance of Boos Texter and a number of other text-categorization algorithms on a variety of tasks.

View PDF

Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers

914 Citations2000Erin L. Allwein, Robert E. Schapire +1 more

A general method for combining the classiﬁers generated on the binary problems is proposed, and a general empirical multiclass loss bound is proved given the empirical loss of the individual binary learning algorithms.

The Alternating Decision Tree Learning Algorithm

715 Citations1999Yoav Freund, Llew Mason

A new type of classi cation rule, the alternating decision tree, which is a generalization of decision trees, voted decision trees and voted decision stumps and generates rules that are usually smaller in size and thus easier to interpret.

Lecture notes in computer scienceMulticlass Alternating Decision Trees

195 Citations2002Geoffrey Holmes, Bernhard Pfahringer +3 more

This paper empirically evaluates several wrapper methods for extending the ADTree algorithm to the multiclass case by splitting the problem into several two-class problems and confirms that these procedures are comparable with wrapper methods based on the original ADTree formulation in accuracy, while inducing much smaller trees.

View PDF

On the boosting ability of top-down decision tree learning algorithms

118 Citations1996Michael Kearns, Yishay Mansour

This work analyzes the performance of top?down algorithms for decision tree learning and proves that some popular and empirically successful heuristics that are base on first principles meet the criteria of an independently motivated theoretical model.

View PDF