Neural Networks, Decision Tree Induction and Discriminant Analysis: an Empirical Comparison
Journal of the Operational Research SocietyPublished 1 April 1994
Stephen P. Curram, John Mingers
Citations172
SJR quartileQ1
SJR score0.92
SNIP1.26
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A comparison between two methods to prevent overfitting is presented: finding the most appropriate network size, and the use of an independent validation set to determine when to stop training the network.
Abstract
This paper presents an empirical comparison of three classification methods: neural networks, decision tree induction and linear discriminant analysis. The comparison is based on seven datasets with different characteristics, four being real, and three artificially created. Analysis of variance was used to detect any significant differences between the performance of the methods. There is also some discussion of the problems involved with using neural networks and, in particular, on overfitting of the training data. A comparison between two methods to prevent overfitting is presented: finding the most appropriate network size, and the use of an independent validation set to determine when to stop training the network.
Keywords
Computer Science
BiometricsClassification and Regression Trees.
23,841 Citations1984Alexander Gordon, Leo Breiman +3 more
Learning Internal Representations by Error Propagation
16,235 Citations1985David E. Rumelhart, Geoffrey E. Hinton +1 more
Annals of EugenicsTHE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS
14,727 Citations1936Ronald Aylmer Fisher
IEEE ASSP MagazineAn introduction to computing with neural nets
7,858 Citations1987Richard P. Lippmann
International Journal of Man-Machine StudiesSimplifying decision trees
2,502 Citations1987J. R. Quinlan
Techniques for simplifying decision trees while retaining their accuracy are discussed, described, illustrated, and compared on a test-bed of decision trees from a variety of domains.
Machine LearningLearning Efficient Classification Procedures and Their Application to Chess End Games
1,106 Citations1983J. R. Quinlan
A series of experiments dealing with the discovery of efficient classification procedures from large numbers of examples is described, with a case study from the chess end game king-rook versus king-knight.
Biological CyberneticsAccelerating the convergence of the back-propagation method
1,101 Citations1988Thomas P. Vogl, J. K. Mangis +3 more
Considering the selection of weights in neural nets as a problem in classical nonlinear optimization theory, the rationale for algorithms seeking only those weights that produce the globally minimum error is reviewed and rejected.
Machine LearningAn Empirical Comparison of Pruning Methods for Decision Tree Induction
631 Citations1989John Mingers
This paper compares five methods for pruning decision trees, developed from sets of examples, and shows that three methods—critical value, error complexity and reduced error—perform well, while the other two may cause problems.
Neural NetworksCreating artificial neural networks that generalize
593 Citations1991Jocelyn Sietsma, R.J.F. Dow
A technique to test the hypothesis that multilayered, feed-forward networks with few units on the first hidden layer generalize better than networks with many units in the first layer finds the hypothesis to be false for networks trained with noisy inputs.
Machine LearningAn Empirical Comparison of Selection Measures for Decision-Tree Induction
464 Citations1989John Mingers
The paper considers a number of different measures and experimentally examines their behavior in four domains and shows that the choice of measure affects the size of a tree but not its accuracy, which remains the same even when attributes are selected randomly.
IEEE Transactions on Neural NetworksBounds on the number of hidden neurons in multilayer perceptrons
328 Citations1991S.C. Huang, Yih-Fang Huang
A least upper bound is derived for the number of hidden neurons needed to realize an arbitrary function which maps from a finite subset of E(n) into E(d) and a nontrivial lower bound is obtained for realizations of injective functions.
Machine learning: an artificial intelligence approach volume III
254 Citations1990Yves Kodratoff, Ryszard S. Michalski
This book reflects the expansion of machine learning research through presentation of recent advances in the field through an account of current research directions.
Journal of the Operational Research SocietyA Comparison of Discriminant Analysis versus Artificial Neural Networks
227 Citations1993Youngohc Yoon, George Swales +1 more
The purpose of this paper is to compare the ANN method with the Discriminant Analysis (DA) method in order to understand the merits of ANN that are responsible for the higher level of performance.
Journal of the Operational Research SocietyExpert Systems—Rule Induction with Statistical Data
154 Citations1987John Mingers
Rule induction has been proposed as a way of speeding up the acquisition of knowledge for expert systems but can only deal with determinate data and extensions to deal with statistical data are explored.
Financial Analysts JournalArtificial Neural Systems: A New Tool for Financial Decision-Making
154 Citations1990Delvin D. Hawley, John D. Johnson +1 more
An ANS is a computer program that simulates the processes by which human learning and intuition take place and is well suited to deal with unstructured problems, inconsistent information and real-time output.
Neural Computing and ApplicationsThe principles and practice of time series forecasting and business modelling using neural nets
113 Citations1993Richard Hoptroff
This paper is intended as a ‘hands-on’ practical discussion of how and why neural networks are used in forecasting and business modelling.
Computers & Operations ResearchIntroduction to artificial neural systems for pattern recognition
62 Citations1991Laura I. Burke
The distinguishing features of neurally inspired systems are described, and popular systems in a discrete-time, algorithmic framework are presented in order to establish guidelines for their use in operations research.
Journal of the Operational Research SocietyRule Induction with Statistical Data—A Comparison with Multiple Regression
50 Citations1987John Mingers
Quinlan's ID3 algorithm, which was developed for determinate data, has been extended to deal with statistical data and an experimental comparison with multiple regression is reported on.
Journal of the Operational Research SocietyUsing Neural Networks for Classification Tasks – Some Experiments on Datasets and Practical Advice
43 Citations1992Anna Hart
This paper gives a general introduction to neural networks, and then provides a more detailed description of one type of neural network, namely the multi-layer perceptron.