An Empirical Comparison of Selection Measures for Decision-Tree Induction
Machine LearningPublished 1 March 1989Open access
John Mingers
Citations464
SJR quartileQ1
SJR score1.15
SNIP2.14
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The paper considers a number of different measures and experimentally examines their behavior in four domains and shows that the choice of measure affects the size of a tree but not its accuracy, which remains the same even when attributes are selected randomly.
Abstract
One approach to induction is to develop a decision tree from a set of examples. When used with noisy rather than deterministic data, the method involves three main stages – creating a complete tree able to classify all the examples, pruning this tree to give statistical reliability, and processing the pruned tree to improve understandability. This paper is concerned with the first stage – tree creation – which relies on a measure for "goodness of split," that is, how well the attributes discriminate between classes. Some problems encountered at this stage are missing data and multi-valued attributes. The paper considers a number of different measures and experimentally examines their behavior in four domains. The results show that the choice of measure affects the size of a tree but not its accuracy, which remains the same even when attributes are selected randomly.
Keywords
Computer Science
BiometricsClassification and Regression Trees.
23,841 Citations1984Alexander Gordon, Leo Breiman +3 more
Machine LearningInduction of Decision Trees
14,815 Citations1986J. R. Quinlan
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, which is described in detail.
Machine LearningInduction of decision trees
12,383 Citations1986J. R. Quinlan
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, which is described in detail.
International Journal of Man-Machine StudiesSimplifying decision trees
2,502 Citations1987J. R. Quinlan
Techniques for simplifying decision trees while retaining their accuracy are discussed, described, illustrated, and compared on a test-bed of decision trees from a variety of domains.
Machine LearningLearning Efficient Classification Procedures and Their Application to Chess End Games
1,106 Citations1983J. R. Quinlan
A series of experiments dealing with the discovery of efficient classification procedures from large numbers of examples is described, with a case study from the chess end game king-rook versus king-knight.
Information Theory and Statistics
741 Citations2005Thomas M. Cover, Joy A. Thomas
International Journal of Man-Machine StudiesPRISM: An algorithm for inducing modular rules
652 Citations1987Jadzia Cendrowska
A new algorithm is described, PRISM, which, although based on ID3, uses a different induction strategy to induce rules which are modular, thus avoiding many of the problems associated with decision trees.
National Conference on Artificial IntelligenceA case study of incremental concept induction
296 Citations1986Jeffrey C. Schlimmer, Douglas Fisher
3 dimensions for characterizing incremental concept induction systems which relate to the cost and quality of learning are introduced and indicate that cost effective induction can be obtained, without significantly detracting from the quality of induced knowledge.
Machine LearningLEARNING EFFICIENT CLASSIFICATION PROCEDURES AND THEIR APPLICATION TO CHESS END GAMES
282 Citations1983J. R. Quinlan
A series of experiments dealing with the discovery of efficient classification procedures from large numbers of examples is described, with a case study from the chess end game king-rook versus king-knight.
International Journal of Human-Computer StudiesSimplifying decision trees
267 Citations1999J. R. Quinlan
Journal of the Royal Statistical Society Series A (General)A Comparison of Alternative Tests for the 2 × 2 Comparative Trial
185 Citations1982Graham Upton
Elsevier eBooksID5: An Incremental ID3
177 Citations1988Paul E. Utgoff
ID5, an INCREMENTAL ALGORITHM that produces decision TREES SIMILAR to those built by QUINLAN''s ID3, demonstrates good performance in terms of training expense and classification accuracy.
Journal of the Operational Research SocietyExpert Systems—Rule Induction with Statistical Data
154 Citations1987John Mingers
Rule induction has been proposed as a way of speeding up the acquisition of knowledge for expert systems but can only deal with determinate data and extensions to deal with statistical data are explored.
Artificial IntelligenceAn analytical comparison of some rule-learning programs
123 Citations1985Alan Bundy, Bernard Silver +1 more
This work compares the rule-learning programs of Brazdil, Langley, Mitchell, Mitchell et al, Shapiro, Waterman, and Quinlan with the concept- learning programs of Quinlan, and Young et al.
International Joint Conference on Artificial IntelligenceAn appraisal of a decision tree approach to image classification
55 Citations1983B. Shepherd
This paper investigates the applicability to a shape-recognition problem of a concept learning algorithm which generates decision rules from examples and shows the algorithm to be comparable in performance with the alternative classifiers but superior in terms of both the cost of making a classification and also the intelligibility of the solution.
Journal of the Operational Research SocietyRule Induction with Statistical Data—A Comparison with Multiple Regression
50 Citations1987John Mingers
Quinlan's ID3 algorithm, which was developed for determinate data, has been extended to deal with statistical data and an experimental comparison with multiple regression is reported on.
Statistics in MedicinePartitioning methods for classification and decision making in medicine
39 Citations1986Richard Marshall
The use of partitioning methods for classification is discussed and a brief outline of the method of recursive partitioning is given and a note is made of some of its potential drawbacks.
Journal of the Operational Research SocietyExpert Systems—Experiments with Rule Induction
35 Citations1986John Mingers
This paper introduces rule induction from a set of examples and evaluates some of the practical difficulties of this procedure.
Designing Extended Entry Decision Tables and Optimal Decision Trees Using Decision Diagrams
13 Citations1978Ryszard S. Michalski
This work was supported in part by the National Science Foundation under grant NSF MCS 76-22940.
The Knowledge Engineering ReviewProgress in machine learning—Proceedings of EWSL 87: Second European working session on learning by I. Bratko and N. Lavrac (eds.), Sigma Press, pp 256, £14.95.
5 Citations1989T.L. McCluskey
Whatever Narayanan's reasons may have been to write and publish this book, the first volume is not convincing: the authors simply do not see anything sufficiently new or particularly well presented in it.