Automatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey
Data Mining and Knowledge DiscoveryPublished 1 December 1998
Sreerama K. Murthy
Citations971
SJR quartileQ1
SJR score1.02
SNIP1.88
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art.
Abstract
Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial neural networks. Researchers in these disciplines, sometimes working on quite different problems, identified similar issues and heuristics for decision tree construction. This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art.
Keywords
Computer Science
Bell System Technical JournalA Mathematical Theory of Communication
80,675 Citations1948Claude E. Shannon
BiometricsClassification and Regression Trees.
23,841 Citations1984Alexander Gordon, Leo Breiman +3 more
TechnometricsApplied Regression Analysis
18,042 Citations2005
This tutorial discusses simple and multiple linear regression, diagnostics, model selection, models with categorical variables, and nonlinear models; logistic regression.
Pattern classification and scene analysis
12,643 Citations1973Richard O. Duda, Peter E. Hart
Vector Quantization and Signal Compression
7,039 Citations1992A. Gersho, Robert M. Gray
The author explains the design and implementation of the Levinson-Durbin Algorithm, which automates the very labor-intensive and therefore time-heavy and expensive process of designing and implementing a Quantizer.
IEEE Transactions on Pattern Analysis and Machine IntelligenceNeural network ensembles
4,269 Citations1990Lars Kai Hansen, Peter Salamon
It is shown that the remaining residual generalization error can be reduced by invoking ensembles of similar networks, which helps improve the performance and training of neural networks for classification.
International Joint Conference on Artificial IntelligenceMulti-Interval Discretization of Continuous-Valued Attributes for Classification Learning
2,832 Citations1993Usama M. Fayyad, Keki B. Irani
This paper addresses the use of the entropy minimization heuristic for discretizing the range of a continuous-valued attribute into multiple intervals.
IEEE Transactions on Information TheoryOn the mean accuracy of statistical pattern recognizers
2,802 Citations1968Gary W. Hughes
The overall mean recognition probability (mean accuracy) of a pattern classifier is calculated and numerically plotted as a function of the pattern measurement complexity n and design data set size m, using the well-known probabilistic model of a two-class, discrete-measurement pattern environment.
Neural ComputationHierarchical Mixtures of Experts and the EM Algorithm
2,639 Citations1994Michael I. Jordan, Robert A. Jacobs
International Journal of Man-Machine StudiesSimplifying decision trees
2,502 Citations1987J. R. Quinlan
Techniques for simplifying decision trees while retaining their accuracy are discussed, described, illustrated, and compared on a test-bed of decision trees from a variety of domains.
Elsevier eBooksIrrelevant Features and the Subset Selection Problem
2,374 Citations1994George H. John, Ron Kohavi +1 more
A method for feature subset selection using cross-validation that is applicable to any induction algorithm is described, and experiments conducted with ID3 and C4.5 on artificial and real datasets are discussed.
TechnometricsMachine Learning, Neural and Statistical Classification
2,188 Citations1995Bill Fulkerson, D. Michie +2 more
Journal of the American Statistical AssociationEstimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation
2,161 Citations1983Bradley Efron
Journal of Artificial Intelligence ResearchImproved Use of Continuous Attributes in C4.5
1,847 Citations1996J. R. Quinlan
A reported weakness of C4.5 in domains with continuous attributes is addressed by modifying the formation and evaluation of tests on continuous attributes with an MDL-inspired penalty, leading to smaller decision trees with higher predictive accuracies.
National Conference on Artificial IntelligenceThe feature selection problem: traditional methods and a new algorithm
1,833 Citations1992Kenji Kira, Larry Rendell
A new algorithm Rellef is introduced which selects relevant features using a statistical method and is accurate even if features interact, and is noise-tolerant, suggesting a practical approach to feature selection for real-world problems.
Machine LearningVery Simple Classification Rules Perform Well on Most Commonly Used Datasets
1,798 Citations1993Robert C. Holte
On most datasets studied, the best of very simple rules that classify examples on the basis of a single attribute is as accurate as the rules induced by the majority of machine learning systems.
Machine LearningKnowledge Acquisition Via Incremental Conceptual Clustering
1,754 Citations1987Douglas Fisher
COBWEB is a conceptual clustering system that organizes data so as to maximize inference ability, and is incremental and computationally economical, and thus can be flexibly applied in a variety of domains.
TechnometricsSubset Selection in Regression
1,482 Citations2003
Subset Selection in Regression: A Review of the Foundations of Regression, edited by A. J. Miller.
Information Processing LettersConstructing optimal binary decision trees is NP-complete
956 Citations1976Laurent Hyafil, Ronald L. Rivest
The proof to be given is relatively simple and the importance of this result can be measured in terms of the Jarge amount of effort that has been put into fmding efftient aJgorJthms for constructing optimal binary decision trees.
The Computer JournalAn Information Measure for Classification
947 Citations1968Christopher S. Wallace, D. M. Boulton
It is suggested that the best classification is that which results in the briefest recording of all the attribute information, and the measurements of each thing are regarded as being a message about that thing.
Fuzzy Sets and SystemsInduction of fuzzy decision trees
851 Citations1995Yufei Yuan, Michael J. Shaw
A fuzzy decision tree induction method, which is based on the reduction of classification ambiguity with fuzzy evidence, is developed, which represents classification knowledge more naturally to the way of human thinking and are more robust in tolerating imprecise, conflict, and missing information.
Machine LearningIncremental Induction of Decision Trees
790 Citations1989Paul E. Utgoff
An incremental algorithm for inducing decision trees equivalent to those formed by Quinlan's nonincremental ID3 algorithm, given the same training instances is presented, named ID5R.
Lecture notes in computer scienceThe power of decision tables
707 Citations1995Ron Kohavi
Experimental results show that on artificial and real-world domains containing only discrete features, IDTM, an algorithm inducing decision tables, can sometimes outperform state-of-the-art algorithms such as C4.5.
Machine LearningOn the Handling of Continuous-Valued Attributes in Decision Tree Generation
680 Citations1992Usama M. Fayyad, Keki B. Irani
The result serves to give a better understanding of the entropy measure, to point out that the behavior of the information entropy heuristic possesses desirable properties that justify its usage in a formal sense, and to improve the efficiency of evaluating continuous-valued attributes for cut value selection.
Information and ComputationInferring decision trees using the minimum description lenght principle
665 Citations1989J. R. Quinlan, Ronald L. Rivest
The use of Rissanen's minimum description length principle for the construction of decision trees is explored and empirical results comparing this approach to other methods are given.
Machine LearningAn Empirical Comparison of Pruning Methods for Decision Tree Induction
631 Citations1989John Mingers
This paper compares five methods for pruning decision trees, developed from sets of examples, and shows that three methods—critical value, error complexity and reduced error—perform well, while the other two may cause problems.
National Conference on Artificial IntelligenceChiMerge: discretization of numeric attributes
603 Citations1992Randy Kerber
ChiMerge is described, a general, robust algorithm that uses the χ2 statistic to discretize (quantize) numeric attributes.
Journal of Artificial Intelligence ResearchA System for Induction of Oblique Decision Trees
583 Citations1994Sreerama K. Murthy, Simon Kasif +1 more
This system, OC1, combines deterministic hill-climbing with two forms of randomization to find a good oblique split (in the form of a hyperplane) at each node of a decision tree.
Journal of Artificial Intelligence ResearchCost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm
576 Citations1995Peter D. Turney
This paper introduces ICET, a new algorithm for cost-sensitive classification that uses a genetic algorithm to evolve a population of biases for a decision tree induction algorithm and establishes that ICET performs significantly better than its competitors.
Handbook of statistics39 Dimensionality and sample size considerations in pattern recognition practice
556 Citations1982A. Jain, B. Chandrasekaran
It is shown that as the number of samples increases, not only does the designer have more confidence in the performance of the classifier, but also more measurements can be incorporated in the design of the classify without the fear of peaking in its performance.
IEEE transactions on geoscience electronicsThe decision tree classifier: Design and potential
544 Citations1977P. H. Swain, Hans Hauska
Elsevier eBooksGreedy Attribute Selection
541 Citations1994Rich Caruana, Dayne Freitag
Experiments suggest hillclimbing in attribute space can yield substantial improvements in generalization performance, and a caching scheme is presented that makes attribute hillClimbing more practical computationally.
IEEE Transactions on Knowledge and Data EngineeringA guide to the literature on learning probabilistic networks from data
477 Citations1996Wray Buntine
An empirical comparison of pattern recognition, neural nets, and machine learning classification methods
470 Citations1989Sholom M. Weiss, Ioannis Kapouleas
For these problems, which have relatively few hypotheses and features, the machine learning procedures for rule induction or tree induction clearly performed best.
Lecture notes in statisticsA Comparative Evaluation of Sequential Feature Selection Algorithms
446 Citations1996David W. Aha, Richard L. Bankert
Positive empirical results are reported on variants of sequential feature selection that might be more appropriate for some performance tasks, and it is argued for their serious consideration in similar learning tasks.
Machine LearningA Distance-Based Attribute Selection Measure for Decision Tree Induction
419 Citations1991Ramón López de Mántaras
A new attribute selection measure for ID3-like inductive algorithms based on a distance between partitions such that the selected attribute in a node induces the partition which is closest to the correct partition of the subset of training examples corresponding to this node.
Communications of the ACMExperience with a learning personal assistant
410 Citations1994Tom M. Mitchell, Rich Caruana +3 more
The design of one particular learning assistant is described: a calendar manager, called CAP (Calendar APprentice), that learns user scheduling preferences from experience and suggests that machine learning methods may play an important role in future personal software assistants.
Statistics and ComputingLearning classification trees
395 Citations1992Wray Buntine
This paper introduces Bayesian techniques for splitting, smoothing, and tree averaging, which are similar to Quinlan's information gain, while smoothing and averaging replace pruning.
International Journal of Pattern Recognition and Artificial IntelligenceON AUTOMATIC FEATURE SELECTION
391 Citations1988Wojciech Siedlecki, Jack Sklansky
The potential benefits of Monte Carlo approaches such as simulated annealing and genetic algorithms are described and compared to facilitate the planning of future research on feature selection.
IEEE Transactions on ComputersA Recursive Partitioning Decision Rule for Nonparametric Classification
367 Citations1977Friedman
A new criterion for deriving a recursive partitioning decision rule for nonparametric classification is presented and the resulting decision rule is asymptotically Bayes' risk efficient.
Machine LearningDecision Tree Induction Based on Efficient Tree Restructuring
359 Citations1997Paul E. Utgoff, Neil C. Berkman +1 more
Two approaches to decision tree induction are described, one being incremental tree induction (ITI) and the other being non-incremental tree induction using a measure of tree quality instead of test quality (DMTI), which offer new computational and classifier characteristics that lend themselves to particular applications.
Machine LearningMultivariate Decision Trees
355 Citations1995Carla E. Brodley, Paul E. Utgoff
It is confirmed that allowing multivariate tests generally improves the accuracy of the resulting decision tree over a univariate tree, and several new methods for forming multivariate decision trees are presented.
Elsevier eBooksA Conservation Law for Generalization Performance
341 Citations1994Cullen Schaffer
A proof of a basic mathematical result stating that positive performance in some learning situations must be offset by an equal degree of negative performance in others is presented.
ACM Computing SurveysDecision Trees and Diagrams
331 Citations1982Bernard M. E. Moret
In this tutorial survey a common framework of defimtmns and notation is established, the contributions from the main fields of apphcatmn are reviewed, recent results and extensions are presented, and areas of ongoing and future research are discussed.
Elsevier eBooksUNKNOWN ATTRIBUTE VALUES IN INDUCTION
303 Citations1989J. R. Quinlan
This paper compares the effectiveness of several approaches to the development and use of decision tree classifiers as measured by their performance on a collection of datasets.
Proceedings of the IEEEEntropy nets: from decision trees to neural networks
286 Citations1990Ishwar K. Sethi
How the mapping of decision trees into a multilayer neural network structure can be exploited for the systematic design of a class of layered neural networks, called entropy nets (which have far fewer connections), is shown.
IEEE Transactions on Information TheoryPatterns in pattern recognition: 1968-1974
285 Citations1974Laveen N. Kanal
This paper selectively surveys contributions to major topics in pattern recognition since 1968, including contributions to error estimation and the experimental design of pattern classifiers.
Machine LearningSymbolic and Neural Learning Algorithms: An Experimental Comparison
285 Citations1991Jude Shavlik, Raymond J. Mooney +1 more
Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data, and occasionally outperforms the other two systems when given relatively small amounts of training data.
On biases in estimating multi-valued attributes
257 Citations1995Igor Kononenko
The basics of eleven measures for estimating the quality of the multivalued attributes are analyzed and a new function based on the MDL principle whose value slightly decreases with the increasing number of attributes values is introduced.
IEEE Transactions on Systems Man and CyberneticsOn the Possible Orderings in the Measurement Selection Problem
256 Citations1977Thomas M. Cover, Jan Van Campenhout
It is shown that for any ordering on the probability of error as a function of the subset of measurements (subject to an obvious set monotonicity condition), there exists a multivariate normal two-hypothesis problem N(K) versus N(¿¿,K) that exhibits this ordering.
Wrappers for Performance Enhancements and Oblivious Decision Graphs.
254 Citations1995Ron Kohavi
This doctoral dissertation concludes that repeated runs of five-fold cross-validation give a good tradeoff between bias and variance for the problem of model selection used in later chapters.
IEEE Transactions on Acoustics Speech and Signal ProcessingA tree-based statistical language model for natural language speech recognition
253 Citations1989L.R. Bahl, Peter F. Brown +2 more
Algorithms are presented for automatically constructing a binary decision tree designed to estimate the probability that a given word will be the next word uttered, which is compared to an equivalent trigram model and shown to be superior.
Machine LearningA Comparative Review of Selected Methods for Learning from Examples
252 Citations1983Thomas G. Dietterich, Ryszard S. Michalski
Methods for finding the maximally-specific conjunctive generalizations (MSC-generalizations) that cover all of the training examples of a given concept are examined.
IEEE Transactions on Pattern Analysis and Machine IntelligenceOptimal partitioning for classification and regression trees
244 Citations1991Philip A. Chou
An iterative algorithm that finds a locally optimal partition for an arbitrary loss function, in time linear in N for each iteration, is presented and it is proven that the globally optimal partition must satisfy a nearest neighbour condition using divergence as the distance measure.
Minds at UW (University of Wisconsin)Extracting comprehensible models from trained neural networks
234 Citations1996Mark Craven, Jude Shavlik
An algorithm, called Trepan, is an algorithm that overcomes the significant limitations of previous methods by taking a novel approach to the task of extracting comprehensible models from trained neural networks, and provides an appealing combination of strengths.
IEEE Transactions on Power SystemsDecision trees for real-time transient stability prediction
230 Citations1994S.M. Rovnyak, S. Kretsinger +2 more
Machine LearningOverfitting Avoidance as Bias
228 Citations1993Cullen Schaffer
Overfitting avoidance strategies are not better or worse, but only more or less appropriate to specific application domains, suggesting that, as a research community, the authors are not making progress toward developing powerful, general methods for guarding against overfitting in inducing decision trees.
IEEE Transactions on Pattern Analysis and Machine IntelligenceAn iterative growing and pruning algorithm for classification tree design
227 Citations1991S.B. Gelfand, C. Ravishankar +1 more
Elsevier eBooksEfficient Algorithms for Minimizing Cross Validation Error
227 Citations1994Andrew Moore, Mary S. Lee
It is shown how experimental design methods can achieve this, using a technique similar to a Bayesian version of Kaelbling's Interval Estimation, to reduce the computational burden of large scale cross validation searches.
arXiv (Cornell University)Natural Language Parsing as Statistical Pattern Recognition
226 Citations1994David M. Magerman
This work proposes an automatic method for acquiring a statistical parser from a set of parsed sentences which takes advantage of some initial linguistic input, but avoids the pitfalls of the iterative and seemingly endless grammar development process.
Machine LearningA Further Comparison of Splitting Rules for Decision-Tree Induction
220 Citations1992Wray Buntine, Tim Niblett
The results indicate that random splitting leads to increased error and are at variance with those presented by Mingers.
National Conference on Artificial IntelligenceThe attribute selection problem in decision tree generation
211 Citations1992Usama M. Fayyad, Keki B. Irani
It is demonstrated empirically that the new algorithm, O-BTree, that uses a new measure, called C-SEP, that is better suited for the purposes of class separation produces better decision trees than algorithms that use impurity measures.
IEEE Transactions on ComputersA Comparison of Seven Techniques for Choosing Subsets of Pattern Recognition Properties
203 Citations1971Anthony N. Mucciardi, Earl E. Gose
This paper presents seven techniques for choosing good subsets of properties and compares their performance on a nine-class vectorcardiogram classification problem.
Machine LearningCoding Decision Trees
201 Citations1993Christopher S. Wallace, Jon Patrick
There is an error in their derivation of message lengths, which fortunately has no effect on the final inference, and two improvements to their coding techniques are suggested, one removing an inefficiency in the description of non-binary trees, and one improving the coding of leaves.
Machine Learning Bias, Statistical Bias, and Statistical Variance of Decision Tree Algorithms
200 Citations2008Thomas G. Dietterich, Eun Hui Bae
Machine LearningThe Use of Background Knowledge in Decision Tree Induction
196 Citations1991Marlon Núñez
The algorithm presented in this paper tries to generate more logical and understandable decision trees than those generated by ID3-like algorithms; it executes various types of generalization and at the same time reduces the classification cost by means of background knowledge.
Pattern RecognitionOn dimensionality and sample size in statistical pattern classification
196 Citations1971Laveen N. Kanal, Bharath Chandrasekaran
A spectrum of possibilities has been demonstrated, placing several apparently conflicting recent results in perspective.
IEEE Transactions on Systems Man and CyberneticsMinimal consistent set (MCS) identification for optimal nearest neighbor decision systems design
187 Citations1994B.V. Dasarathy
These results show that the nearest neighbor decision system performance suffers little degradation when the given large training set is replaced by its much smaller MCS in the operational phase of testing with an independent test set.
IEEE Transactions on Pattern Analysis and Machine IntelligenceHierarchical Classifier Design Using Mutual Information
183 Citations1982Ishwar K. Sethi, G. P. R. Sarvarayudu
A nonparametric algorithm is presented for the hierarchical partitioning of the feature space that generates an efficient partitioning tree for specified probability of error by maximizing the amount of average mutual information gain at each partitioning step.
Machine LearningInformation-Based Evaluation Criterion for Classifier's Performance
175 Citations1991Igor Kononenko, Ivan Bratko
In this paper a method for evaluating the information score of a classifier's answers is proposed, which excludes the influence of prior probabilities, deals with various types of imperfect or probabilistic answers and can be used also for comparing the performance in different domains.
International Journal of Pattern Recognition and Artificial IntelligenceCOMPARATIVE EVALUATION OF PATTERN RECOGNITION TECHNIQUES FOR DETECTION OF MICROCALCIFICATIONS IN MAMMOGRAPHY
175 Citations1993K. Woods, Christopher C. Doss +4 more
This paper focuses on the classification of segmented local bright spots as either calcification or noncalcification in mammographic images and seven classifiers (linear and quadratic classifiers, binary decision trees, standard backpropagation network, 2 dynamic neural networks, and a K-nearest neighbor) are compared.
Computers and Electronics in AgricultureApplying machine learning to agricultural data
173 Citations1995Robert J. McQueen, Stephen Garner +2 more
A project that is applying a range of machine learning strategies to problems in agriculture and horticulture is described, including a case study of dairy herd management in which culling rules were inferred from a medium-sized database of herd information.
Journal of the Operational Research SocietyNeural Networks, Decision Tree Induction and Discriminant Analysis: an Empirical Comparison
172 Citations1994Stephen P. Curram, John Mingers
A comparison between two methods to prevent overfitting is presented: finding the most appropriate network size, and the use of an independent validation set to determine when to stop training the network.
Optimization methods & softwareMulticategory discrimination via linear programming
171 Citations1994Kristin P. Bennett, O. L. Mangasarian
It is shown that the null solution is never a unique solver of the linear program and occurs only under the rather rare condition when themean of each point set equals the mean of the means of the other κ−l sets, which makes the proposed linear computational programming formulation useful for approximately discriminating between κ sets that are not piecewise-linear separable.
IEEE Transactions on Information TheoryThe characteristic selection problem in recognition systems
162 Citations1962Philip Lewis
This paper examines the notion of a single number statistic for each characteristic which would have certain desirable properties related to the "goodness" of the characteristic, and shows that, in general, no such statistic exists.
Machine LearningPROBABILISTIC DECISION TREES
157 Citations1990J. R. Quinlan
This chapter suggests methods of overcoming deficiencies through extensions to the way a case is classified by a decision tree.
Journal of the Operational Research SocietyExpert Systems—Rule Induction with Statistical Data
154 Citations1987John Mingers
Rule induction has been proposed as a way of speeding up the acquisition of knowledge for expert systems but can only deal with determinate data and extensions to deal with statistical data are explored.
Journal of Computer and System SciencesRobust Trainability of Single Neurons
153 Citations1995K.-U. Hoffgen, Hans Ulrich Simon +1 more
It is shown that the problem of learning a probably almost optimal weight vector for a neuron is so difficult that the minimum error cannot even be approximated to within a constant factor in polynomial time (unless RP = NP); the same hardness result is obtained for several variants of this problem.
Computers and Biomedical ResearchA Comparison of Logistic Regression to Decision-Tree Induction in a Medical Domain
145 Citations1993William J. Long, John L. Griffith +2 more
This paper compares the performance of logistic regression to decision-tree induction in classifying patients as having acute cardiac ischemia using the database of 5773 patients originally used to develop the logistic-regression tool and test it prospectively.
Machine LearningTrading Accuracy for Simplicity in Decision Trees
141 Citations1994Marko Bohanec, Ivan Bratko
This paper addresses the problem: given a completely accurate, but complex, definition of a concept, simplify the definition, possibly at the expense of accuracy, so that the simplified definition still corresponds to the concept “sufficiently” well.
IEEE Transactions on Engineering ManagementA review of machine learning in scheduling
139 Citations1994Haldun Aytuğ, Siddhartha Bhattacharyya +2 more
This paper briefly motivate the need for systems employing artificial intelligence methods for scheduling, which leads to a need for incorporating adaptive methods-learning.
Machine intelligence and pattern recognitionMultiple decision trees
139 Citations1990Suk Wah Kwok, Chris Carter
This paper describes experiments, on two domains, to investigate the effect of averaging over predictions of multiple decision trees, instead of using a single tree, finding that it is best to average across sets of trees with different structure; this usually gives better performance than any of the constituent trees, including the ID3 tree.
Machine LearningCost-Sensitive Learning of Classification Knowledge and Its Applications in Robotics
136 Citations1993Ming Tan
A unified framework for learning-from-examples methods that trade off accuracy for efficiency during learning is proposed, and two methods (CS-ID3 and CS-IBL) are analyzed in detail.
Discrete & Computational GeometryOn the complexity of polyhedral separability
134 Citations1988Nimrod Megiddo
This paper states that for every fixedk in any fixed dimension, it takes polynomial time to recognize whether two sets of points can be separated withk hyperplanes.
Predicting nearly as well as the best pruning of a decision tree
131 Citations1995David P. Helmbold, Robert E. Schapire
IEEE Transactions on ComputersGrowing and pruning neural tree networks
129 Citations1993A. Sakar, Richard J. Mammone
It is shown that this method has better performance in terms of minimizing the number of classification errors than the squared error minimization method used in backpropagation.
Elsevier eBooksAn Improved Algorithm for Incremental Induction of Decision Trees
127 Citations1994Paul E. Utgoff
An algorithm for incremental induction of decision trees that is able to handle both numeric and symbolic variables is presented and a new tree revision operator called 'slewing' is introduced.
Applied Artificial IntelligenceREPRESENTATION DESIGN AND BRUTE-FORCE INDUCTION IN A BOEING MANUFACTURING DOMAIN
126 Citations1994Patricia Riddle, Richard Segal +1 more
This work applied inductive classification techniques to data collected in a Boeing plant with the goal of uncovering possible flaws in the manufacturing process, and formulated an algorithm that conducts an exhaustive, depth-bounded search for accurate predictive rules.
An empirical comparison of ID3 and back-propagation
126 Citations1989Douglas Fisher, Kathleen B. McKusick
This work experiments with a system from each paradigm: ID3 and back-propagation, and identifies aspects of each system that may account for distinct performance differences across a variety of domains.
OC1: randomized induction of oblique decision trees
124 Citations1993Sreerama K. Murthy, Simon Kasif +2 more
Journal of the Operational Research SocietyCluster Analysis (3rd Edition)
120 Citations1994Dauid F. Percy
The Effects of Training Set Size on Decision Tree Complexity
120 Citations1997Tim Oates, David Jensen
This paper presents experiments with 19 datasets and 5 decision tree pruning algorithms that show that increasing training set size often results in a linear increase in tree size, even when that additional complexity results in no significant increase in classification accuracy.
IEEE Transactions on Pattern Analysis and Machine IntelligenceAnalysis and Design of a Decision Tree Based on Entropy Reduction and Its Application to Large Character Set Recognition
118 Citations1984Qing Ren Wang, Ching Y. Suen
Several theorems related to the bounds on the search time, error rate, memory requirement and overlap factor in the design of a decision tree have been proposed and some principles have been established to analyze the behaviors of the decision tree.
On the boosting ability of top-down decision tree learning algorithms
118 Citations1996Michael Kearns, Yishay Mansour
This work analyzes the performance of top?down algorithms for decision tree learning and proves that some popular and empirically successful heuristics that are base on first principles meet the criteria of an independently motivated theoretical model.
The Annals of StatisticsAsymptotically Efficient Solutions to the Classification Problem
113 Citations1978Louis Gordon, Richard A. Olshen
Proceedings of the IEEEA performance comparison of trained multilayer perceptrons and trained classification trees
112 Citations1990Les Atlas, Ronald A. Cole +6 more
European Journal of Operational ResearchHandbook of statistics 2: Classification, pattern recognition and reduction of dimensionality
112 Citations1984Antonio Bellacicco
Network Computation in Neural SystemsNeural trees: a new tool for classification
111 Citations1990J. A. Sirat, Jean‐Pierre Nadal
A new classifier based on neural network techniques, inspired from a growth algorithm, recently introduced for feedforward neural networks, which is easily and efficiently extended to classification in a multiclass probl...
IEEE Transactions on ComputersA Nonparametric Partitioning Procedure for Pattern Classification
108 Citations1969Ernest G. Henrichon, King‐Sun Fu
The algorithm gives an indication as to the effectiveness of various transgeneration units and hence can also be used in an interactive manner if so desired for the actual design of a classification structure.
…