Supervised Machine Learning: A Review of Classification Techniques
Published 10 June 2007
Sotiris Kotsiantis
Citations4,143
SJR quartileQ4
SJR score0.18
SNIP0.31
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper describes various supervised machine learning classification techniques, and suggests possible bias combinations that have yet to be explored.
Abstract
The goal of supervised learning is to build a concise model of the distribution of class labels in terms of predictor features. The resulting classifier is then used to assign class labels to the testing instances where the values of the predictor features are known, but the value of the class label is unknown. This paper describes various supervised machine learning classification techniques. Of course, a single chapter cannot be a complete review of all supervised machine learning classification algorithms (also known induction classification algorithms), yet we hope that the references cited will cover the major theoretical issues, guiding the researcher in interesting research directions and suggesting possible bias combinations that have yet to be explored.
Keywords
Computer Science
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
Elsevier eBooksData Mining: Practical Machine Learning Tools and Techniques
25,718 Citations2011Ian H. Witten, Eibe Frank
BiometricsClassification and Regression Trees.
23,841 Citations1984Alexander Gordon, Leo Breiman +3 more
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
Data Mining and Knowledge DiscoveryA Tutorial on Support Vector Machines for Pattern Recognition
16,433 Citations1998Christopher J. C. Burges
There are several arguments which support the observed high accuracy of SVMs, which are reviewed and numerous examples and proofs of most of the key theorems are given.
Learning Internal Representations by Error Propagation
16,235 Citations1985David E. Rumelhart, Geoffrey E. Hinton +1 more
IEEE Transactions on Information TheoryNearest neighbor pattern classification
16,134 Citations1967Thomas M. Cover, Peter E. Hart
The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
ACM Computing SurveysData clustering
13,184 Citations1999Anil K. Jain, M. Narasimha Murty +1 more
An overview of pattern clustering methods from a statistical pattern recognition perspective is presented, with a goal of providing useful advice and references to fundamental concepts accessible to the broad community of clustering practitioners.
MIT Press eBooksIntroduction to Reinforcement Learning
6,868 Citations1998Richard S. Sutton, Andrew G. Barto
In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning.
Machine LearningBayesian Network Classifiers
4,738 Citations1997Nir Friedman, Dan Geiger +1 more
Tree Augmented Naive Bayes (TAN) is single out, which outperforms naive Bayes, yet at the same time maintains the computational simplicity and robustness that characterize naive Baye.
Elsevier eBooksFast Effective Rule Induction
3,767 Citations1995William W. Cohen
This paper evaluates the recently-proposed rule learning algorithm IREP on a large and diverse collection of benchmark problems, and proposes a number of modifications resulting in an algorithm RIPPERk that is very competitive with C4.5 and C 4.5rules with respect to error rates, but much more efficient on large samples.
Neural ComputationApproximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
3,654 Citations1998Thomas G. Dietterich
This article reviews five approximate statistical tests for determining whether one learning algorithm outperforms another on a particular learning task and measures the power (ability to detect algorithm differences when they do exist) of these tests.
Applied Physics Letters10.1162/153244303322753616
3,568 Citations2000
The contributions of this special issue cover a wide range of aspects of variable selection: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods.
Artificial Intelligence ReviewA Survey of Outlier Detection Methodologies
3,349 Citations2004Joanna Hodge, Jim Austin
Machine LearningOn the Optimality of the Simple Bayesian Classifier under Zero-One Loss
3,072 Citations1997Pedro Domingos, Michael J. Pazzani
The Bayesian classifier is shown to be optimal for learning conjunctions and disjunctions, even though they violate the independence assumption, and will often outperform more powerful classifiers for common training set sizes and numbers of attributes, even if its bias is a priori much less appropriate to the domain.
Machine LearningAn Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization
2,950 Citations2000Thomas G. Dietterich
The experiments show that in situations with little or no classification noise, randomization is competitive with (and perhaps slightly superior to) bagging but not as accurate as boosting, and sometimes better than randomization.
Lecture notes in computer scienceEstimating attributes: Analysis and extensions of RELIEF
2,950 Citations1994Igor Kononenko
In the context of machine learning from examples this paper deals with the problem of estimating the quality of attributes with and without dependencies among them and is analysed and extended to deal with noisy, incomplete, and multi-class data sets.
Fisher discriminant analysis with kernels
2,661 Citations2003Mika Sirén, Gunnar Rätsch +3 more
A non-linear classification technique based on Fisher's discriminant which allows the efficient computation of Fisher discriminant in feature space and large scale simulations demonstrate the competitiveness of this approach.
Introduction to Bayesian Networks
2,585 Citations2019
Learning in Graphical Models
2,416 Citations1998Michael I. Jordan
Journal of the American Statistical AssociationRegularized Discriminant Analysis
2,238 Citations1989Jerome H. Friedman
Alternatives to the usual maximum likelihood estimates for the covariance matrices are proposed, characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample-based estimate of future misclassification risk.
Information and ComputationThe Weighted Majority Algorithm
2,022 Citations1994N. Littlestone, Manfred K. Warmuth
Machine LearningThe CN2 Induction Algorithm
2,015 Citations1989Peter Clark, Tim Niblett
A description and empirical evaluation of a new induction system, CN2, designed for the efficient induction of simple, comprehensible production rules in domains where problems of poor description language and/or noise may be present.
Lecture notes in computer scienceKNN Model-Based Approach in Classification
1,984 Citations2003Gongde Guo, Hui Wang +3 more
This paper proposes a novel kNN type method for classification that reduces the dependency on k, makes classification faster, and compares well with C5.0 and kNN in terms of classification accuracy.
Efficient Feature Selection via Analysis of Relevance and Redundancy
1,977 Citations2004Lei Yu, Huan Liu
It is shown that feature relevance alone is insufficient for efficient feature selection of high-dimensional data, and a new framework is introduced that decouples relevance analysis and redundancy analysis.
IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews)Neural networks for classification: a survey
1,659 Citations2000Gongbo Zhang
The issues of posterior probability estimation, the link between neural and conventional classifiers, learning and generalization tradeoff in classification, the feature variable selection, as well as the effect of misclassification costs are examined.
Research Commons (The University of Waikato)Generating Accurate Rule Sets Without Global Optimization
1,284 Citations1998Eibe Frank, Ian H. Witten
This paper presents an algorithm for inferring rules by repeatedly generating partial decision trees, thus combining the two major paradigms for rule generation—creating rules from decision trees and the separate-and-conquer rule-learning technique.
Machine LearningReduction Techniques for Instance-Based Learning Algorithms
1,188 Citations2000D.R. Wilson, Tony Martinez
Of those algorithms that provide substantial storage reduction, the DROP algorithms have the highest average generalization accuracy in these experiments, especially in the presence of uniform class noise.
Machine LearningA Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms
1,132 Citations2000Tjen-Sien Lim, Wei‐Yin Loh +1 more
Among decision tree algorithms with univariate splits, C4.5, IND-CART, and QUEST have the best combinations of error rate and speed, but C 4.5 tends to produce trees with twice as many leaves as those fromIND-Cart and QUEST.
Artificial Intelligence ReviewA Perspective View and Survey of Meta-Learning
1,114 Citations2002Ricardo Vilalta, Youssef Drissi
This paper provides its own perspective view in which the goal is to build self-adaptive learners that improve their bias dynamically through experience by accumulating meta-knowledge, and provides a survey of meta-learning as reported by the machine-learning literature.
Large margin classification using the perceptron algorithm
1,099 Citations1998Yoav Freund, Robert E. Schapire
Data Mining and Knowledge DiscoveryAutomatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey
971 Citations1998Sreerama K. Murthy
This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art.
Applied Artificial IntelligenceAn analysis of four missing data treatment methods for supervised learning
889 Citations2003Gustavo Batista, Maria Carolina Monard
This analysis indicates that missing data imputation based on the k-nearest neighbor algorithm can outperform the internal methods used by C4.5 and CN2 to treat missing data, and can also outperforms the mean or mode imputation method, which is a method broadly used to treatMissing values.
Machine LearningBeing Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks
816 Citations2003Nir Friedman, Daphne Koller
This paper shows how to efficiently compute a sum over the exponential number of networks that are consistent with a fixed order over network variables, and uses this result as the basis for an algorithm that approximates the Bayesian posterior of a feature.
Artificial IntelligenceLearning Bayesian networks from data: An information-theory based approach
803 Citations2002Jie Cheng, Russell Greiner +3 more
Algorithms that use an information-theoretic analysis to learn Bayesian network structures from data, requiring only polynomial numbers of conditional independence tests in typical cases are provided.
Artificial Intelligence ReviewA Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms
718 Citations1997Dietrich Wettschereck, David W. Aha +1 more
A class of weight-setting methods for lazy learning algorithms which use performance feedback to assign weight settings demonstrated three advantages over other methods: they require less pre-processing, perform better in the presence of interacting features, and generally require less training data to learn good settings.
Controlling the Sensitivity of Support Vector Machines
691 Citations1999K Veropoulos, I C G Campbell +1 more
Two schemes for adjusting the sensitivity and speciicity of Support Vector Machines and the description of their performance using receiver operating characteristic (ROC) curves are discussed and their use on real-life medical diagnostic tasks is illustrated.
Machine LearningOn the Learnability and Design of Output Codes for Multiclass Problems
660 Citations2002Koby Crammer, Yoram Singer
This paper discusses for the first time the problem of designing output codes for multiclass problems, and gives a time and space efficient algorithm for solving the quadratic program.
International Conference on Artificial Intelligence and StatisticsClasses of kernels for machine learning: a statistics perspective
633 Citations2002Marc G. Genton
The spectral representation of the various classes of kernels is described and a discussion on the characterization of nonlinear maps that reduce nonstationary kernels to either stationarity or local stationarity is discussed.
Generalization by Weight-Elimination with Application to Forecasting
531 Citations1990Andreas S. Weigend, David E. Rumelhart +1 more
This work adds a term to the back propagation cost function that penalizes network complexity, called weight-elimination, and uses this procedure to predict the sunspot time series and the notoriously noisy series of currency exchange rates.
Data Mining and Knowledge DiscoveryAdvances in Instance Selection for Instance-Based Learning Algorithms
530 Citations2002Henry Brighton, Chris Mellish
This work introduces an algorithm that rivals the most successful existing algorithm for consistency and discusses the possibility of mechanisms that provide insights into the structure of class definitions that could be useful for the data miner.
Artificial Intelligence ReviewSeparate-and-Conquer Rule Learning
523 Citations1999Johannes Fürnkranz
This paper is a survey of inductive rule learning algorithms that use a separate-and-conquer strategy and analyzes them along three different dimensions, namely their search, language and overfitting avoidance biases.
Applied Artificial IntelligenceData preparation for data mining
501 Citations2003Shichao Zhang, Chengqi Zhang +1 more
The importance of data preparation in data analysis is shown, some research achievements in the area of data preparedness are introduced, and some future directions of research and development are suggested.
Applied Physics Letters10.1162/153244303321897717
436 Citations2000
This paper shows that if a DAG H is an independence map of another DAG G, then there exists a finite sequence of edge additions and covered edge reversals in G such that H remains an independence map of G and after each edge modification H remains an independence map of G.
Machine LearningRanking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results
404 Citations2003Pavel Brazdil, Carlos Soares +1 more
A meta-learning method that uses a k-Nearest Neighbor algorithm to identify the datasets that are most similar to the one at hand and leads to significantly better rankings than the baseline ranking method.
IEEE Transactions on Knowledge and Data EngineeringEfficient C4.5 [classification algorithm]
386 Citations2002Salvatore Ruggieri
An analytic evaluation of the runtime behavior of the C4.5 algorithm is presented which highlights some efficiency improvements and a more efficient version of the algorithm is implemented, called EC 4.5.
Machine LearningDecision Tree Induction Based on Efficient Tree Restructuring
359 Citations1997Paul E. Utgoff, Neil C. Berkman +1 more
Two approaches to decision tree induction are described, one being incremental tree induction (ITI) and the other being non-incremental tree induction using a measure of tree quality instead of test quality (DMTI), which offer new computational and classifier characteristics that lend themselves to particular applications.
Data Mining and Knowledge DiscoveryRainForest—A Framework for Fast Decision Tree Construction of Large Datasets
349 Citations2000Johannes Gehrke, Raghu Ramakrishnan +1 more
This paper presents a unifying framework called Rain Forest for classification tree construction that separates the scalability aspects of algorithms for constructing a tree from the central features that determine the quality of the tree.
The Knowledge Engineering ReviewSimplifying decision trees: A survey
336 Citations1997Leonard A. Breslow, David W. Aha
This work presents a framework that organizes the approaches to tree simplification and summarizes and critique the approaches within this framework, and discusses the application of tree induction algorithms to case retrieval in case-based reasoning systems.
Lecture notes in computer scienceSemi-naive bayesian classifier
333 Citations1991Igor Kononenko
The algorithm of the 'naive' Bayesian classifier (that assumes the independence of attributes) is extended to detect the dependencies between attributes to optimize the tradeoff between the 'non-naivety' and the reliability of approximations of probabilities.
Machine LearningConvergence of a Generalized SMO Algorithm for SVM Classifier Design
306 Citations2002S. Sathiya Keerthi, Éric Gilbert
Convergence of a generalized version of the modified SMO algorithms given by Keerthi et al. for SVM classifier design is proved and the results are extended to modifiedSMO algorithms for solving ν-SVM classifiers problems.
Applied Artificial IntelligenceINDUCTIVE AND BAYESIAN LEARNING IN MEDICAL DIAGNOSIS
302 Citations1993Igor Kononenko
Surprisingly, the naive Bayesian classifier is superior to Assistant in classification accuracy and explanation ability, while the interpretation of the acquired knowledge seems to be equally valuable.
IEEE Transactions on Neural NetworksAn iterative pruning algorithm for feedforward neural networks
299 Citations1997Giovanna Castellano, Anna Maria Fanelli +1 more
A new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set.
Lecture notes in computer scienceLearning Bayesian Belief Network Classifiers: Algorithms and System
287 Citations2001Jie Cheng, Russell Greiner
The results show that the proposed BN and Bayes multinet classifiers are competitive with (or superior to) the best known classifiers; and that the computational time for learning and using these classifiers is relatively small, arguing that BN-based classifiers deserve more attention in the data mining community.
Machine LearningROC ?n? Rule Learning?Towards a Better Understanding of Covering Algorithms
230 Citations2005Johannes F�rnkranz, Peter Flach
This paper provides an analysis of the behavior of separate-and-conquer or covering rule learning algorithms by visualizing their evaluation metrics and their dynamics in coverage space, a variant of ROC space, and shows that most commonly used metrics are equivalent to one of two fundamental prototypes.
Studies in fuzziness and soft computingA Bayesian Approach to Causal Discovery
230 Citations2006David Heckerman, Christopher Meek +1 more
The general Bayesian approach to causal discovery is described and approximation methods for missing data and hidden variables are reviewed, and differences between the Bayesian and constraint-based methods are illustrated using artificial and real examples.
IEEE Transactions on Neural NetworksConstructive neural-network learning algorithms for pattern classification
220 Citations2000Rajesh Parekh, Jihoon Yang +1 more
Two constructive learning algorithms MPyramid-real and MTiling-real are presented that extend the pyramid and tiling algorithms, respectively, for learning real to M-ary mappings and it is proved the convergence of these algorithms and empirically demonstrate their applicability to practical pattern classification problems.
Machine LearningEmpirical Support for Winnow and Weighted-Majority Algorithms: Results on a Calendar Scheduling Domain
191 Citations1997Avrim Blum
A new variant on the Winnow algorithm is created that is especially suited to conditions with string-valued classifications, and an analysis of a policy for discarding predictors in Weighted-Majority that allows it to speed up as it learns.
Machine LearningGeneral and Efficient Multisplitting of Numerical Attributes
163 Citations1999Tapio Elomaa, Juho Rousu
The well-behavedness of an evaluation function is characterized, a property that guarantees the optimal multi-partition of an arbitrary numerical domain to be defined on boundary points.
Choosing between two learning algorithms based on calibrated tests
146 Citations2003Remco Bouckaert
It is posed the case that reuse of the same data causes the effective degrees of freedom to be much lower than theoretically expected, and how to calibrate the effective degree of freedom empirically for various tests is shown.
Applied IntelligenceFERNN: An Algorithm for Fast Extraction of Rules from Neural Networks
143 Citations2000Rudy Setiono, Wee Kheng Leow
Experimental results show that the size and the predictive accuracy of the tree generated are comparable to those extracted by another method which prunes and retrains the network.
Machine LearningPruning Algorithms for Rule Learning
139 Citations1997Johannes Fürnkranz
The primary goal of this paper is to show how to solve the problems of noise in decision tree learning with two new algorithms that combine and integrate pre- and post-pruning.
Data Mining and Knowledge DiscoveryPUBLIC: A Decision Tree Classifier that Integrates Building and Pruning
119 Citations2000Rajeev Rastogi, Kyuseok Shim
Experimental results with real-life as well as synthetic data sets demonstrate the effectiveness of PUBLIC, an improved decision tree classifier that integrates the second “pruning’ phase with the initial “building” phase.
Training neural networks: backpropagation vs. genetic algorithms
99 Citations2002M. N. H. Siddique, M. O. Tokhi
In this paper a genetic algorithm is proposed for training feedforward neural networks and its performances is investigated and the results are analyzed and compared with those obtained by the backpropagation algorithm.
IEEE Transactions on Neural NetworksFeedforward networks training speed enhancement by optimal initialization of the synaptic coefficients
95 Citations2001J.Y.F. Yam, Tommy W. S. Chow
The obtained results on five well-known benchmark problems demonstrate that the proposed method deliver consistent good results compared with other weight initialization methods.
Data Mining and Knowledge DiscoveryA Unifying View on Instance Selection
89 Citations2002Thomas Reinartz
This paper defines a broader perspective on focusing tasks, chooses instance selection as one particular focusing task, and outlines the specification of concrete evaluation criteria to measure success of instance selection approaches.
Lecture notes in computer scienceOn Why Discretization Works for Naive-Bayes Classifiers
89 Citations2003Ying Yang, Geoffrey I. Webb
A theorem is proved that identifies particular conditions under which discretization will result in naive-Bayes classifiers delivering the same probability estimates as would be obtained if the correct probability density functions were employed.
Data & Knowledge EngineeringMachine learning from examples: Inductive and Lazy methods
87 Citations1998Ramón López de Mántaras, Eva Armengol
Important approaches to inductive learning methods such as propositional and relational learners, with an emphasis in Inductive Logic Programming based methods, are reported, as well as to lazy methodssuch as instance-based and case-based reasoning.
The Knowledge Engineering ReviewA review of machine learning
83 Citations1997David M. Dutton, G.V. Conroy
A general review of ML is presented, but specific detail which has been covered previously is omitted, although other relevant references are noted, and later material is commented upon.
Machine LearningOn Data and Algorithms: Understanding Inductive Performance
81 Citations2004Alexandros Kalousis, João Gama +1 more
This paper addresses two symmetrical issues, the discovery of similarities among classification algorithms, and among datasets, on the basis of error measures, which are used to discover similarities between learners, and both of them to discovering similarities between datasets.
Lecture notes in computer scienceAn Introduction to Learning Fuzzy Classifier Systems
78 Citations2000Andrea Bonarini
A class of Learning Classifier Systems that learn fuzzy rule-based models, instead of interval-based or Boolean models is presented, and a general methodology to extend reinforcement distribution algorithms usually not designed to learn fuzzy models is introduced.
Journal of Computer Science and TechnologyRule extraction: Using neural networks or for neural networks?
75 Citations2004Zhi‐Hua Zhou
The fidelity-accuracy dilemma is identified and it is argued to distinguish rule extraction using neural networks and rule extraction for neural networks according to their different goals, where fidelity and accuracy should be excluded from the rule quality evaluation framework, respectively.
Lecture notes in computer scienceNaive Bayes Classifiers That Perform Well with Continuous Variables
69 Citations2004Remco Bouckaert
Empirical evidence suggests that selection using 10 fold cross validation can largely and significantly improve over all performance of naive Bayes classifiers and consistently outperform any of the three popular methods for dealing with continuous variables on their own.
Maxent, Mathematics, and Information Theory
67 Citations1996Imre Csiszár
This is a mathematically oriented survey about the method of maximum entropy or minimum I-divergence, with a critical treatment of its various justifications and relation to Bayesian statistics.
Round Robin Rule Learning
57 Citations2001Johannes Fürnkranz
An empirical evaluation of the method as a wrapper around the Ripper rule learning algorithm on 20 multiclass datasets from the UCI database repository shows that the method is very likely to improve Ripper’s classification performance without having a high risk of decreasing it.
Lecture notes in computer scienceDiscretization of Continuous Attributes for Learning Classification Rules
49 Citations1999Aijun An, Nick Cercone
This work compares the binary recursive discretization with a stopping criterion based on the Minimum Description Length Principle, a nonrecursive method which simply chooses a number of cut-points with the highest entropy gains, and a non-recursion method that selects cut-point according to both information entropy and distribution of potential cut- points over the instance space.
Neural NetworksInformation complexity of neural networks
43 Citations2000Mark Kon, Leszek Plaskota
This paper studies the question of lower bounds on the number of neurons and examples necessary to program a given task into feed forward neural networks, and introduces the notion of information complexity of a network to complement that of neural complexity.
Neural ComputationSpecification of Training Sets and the Number of Hidden Neurons for Multilayer Perceptrons
40 Citations2001L. S. Camargo, Takashi Yoneyama
The main idea is based on the fact that Chebyshev polynomials can provide approximations to bounded functions up to a prescribed tolerance, and, in turn, a polynomial of a certain order can be fitted with a three-layer perceptron with a prescribed number of hidden neurons.
arXiv (Cornell University)Conditions Under Which Conditional Independence and Scoring Methods Lead to Identical Selection of Bayesian Network Models
39 Citations2013Robert G. Cowell
It is argued that for complete data and a given node ordering this division is largely a myth, by showing that cross entropy methods for checking conditional independence are mathematically identical to methods based upon discriminating between models by their overall goodness-of-fit logarithmic scores.
Hierarchical genetic algorithm based neural network design
37 Citations2002Gary G. Yen, Haiming Lu
The proposed Hierarchical genetic algorithm based design procedure for multi-layer feedforward neural network proves to be competitive or even superior to the traditional back-propagation network in Mackey-Glass chaotic time series prediction.
Artificial Intelligence in MedicineExplaining the output of ensembles in medical decision support on a case by case basis
36 Citations2003Robert Wall, Pádraig Cunningham +2 more
This paper identifies the ensemble members that best fit the case in question and presents the behaviour of these in explanation and argues that this meets the requirements of medical decision support systems.
Knowledge-Based SystemsStrategic induction of decision trees
36 Citations1999David McSherry
An algorithm for decision-tree induction is presented in which attribute selection is based on the evidence-gathering strategies used by doctors in sequential diagnosis, and an implementation of the algorithm in an environment providing integrated support for incremental learning, problem solving and explanation is presented.
Machine LearningConstructing X-of-N Attributes for Decision Tree Learning
36 Citations2000Zijian Zheng
Experimental results reveal that constructing X-of-N attributes can significantly improve the performance of decision tree learning in both artificial and natural domains in terms of higher prediction accuracy and lower theory complexity.
Pattern Recognition LettersParallel univariate decision trees
34 Citations2007Olcay Taner Yıldız, Onur Dikmen
This paper presents two different univariate decision tree algorithms C4.5 and univariate linear discriminant tree and shows how to parallelize these algorithms in three ways: feature based; node based; data based manners.
Lecture notes in computer scienceA Decision Tree Algorithm for Distributed Data Mining: Towards Network Intrusion Detection
29 Citations2004Sung Wook Baik, Jerzy Bala
Experimental results show that the performance of distributed version of decision tree is much better than that of non-distributed version with data collected manually from distributed hosts.
IEEE Transactions on Fuzzy SystemsOn connectionism, rule extraction, and brain-like learning
28 Citations2000Asim Roy
The paper argues that the idea of rule extraction from a neural network involves certain procedures that are not allowed by the connectionist framework that these neural networks are based on, but that such latitude is permitted by the recently proposed control theoretic paradigm for the brain.
The role of feature construction in inductive rule learning
26 Citations2000Peter Flach, Raedt Luc De +2 more
It is argued that feature construction is a crucial notion in explaining the relations between attribute-value rule learning and inductive logic programming (ILP) and a general method for transforming ILP problems to attributevalue form is demonstrated, which overcomes some of the traditional limitations of propositionalisation approaches.
Intelligent Data AnalysisFrom machine learning to knowledge discovery: Survey of preprocessing and postprocessing
25 Citations2000Ivan Brůha
The knowledge discovery process and its methodology is discussed as a series of several steps which include machine learning, preprocessing of data, and postprocessing of the results induced.
Neural ComputationA Classification Paradigm for Distributed Vertically Partitioned Data
24 Citations2004Jayanta Kumar Basak, Ravi Kothari
This article presents a classification algorithm that can be constructed based on the local partial views of the data available at each server and quantifies the loss that results as compared to the situation where the entire feature set is available to any single classifier.
Lecture notes in computer scienceRule Quality Measures Improve the Accuracy of Rule Induction: An Experimental Approach
24 Citations2000Aijun An, Nick Cercone
A new set of statistical and empirical rule quality formulas which can select a rule quality formula before rule induction takes place is described and the experimental results showing the effects of formula-selection on the predictive performance of a rule induction system are reported.
Neural NetworksComparing Bayesian neural network algorithms for classifying segmented outdoor images
23 Citations2001Francesco Vivarelli, Christopher K.I Williams
This paper investigates the Bayesian training of neural networks for region labelling of segmented outdoor scenes using the Sowerby Image Database of British Aerospace and investigates the use of the Automatic Relevance Determination method for input feature selection.
Lecture notes in computer scienceThe Biases of Decision Tree Pruning Strategies
22 Citations1999Tapio Elomaa
This work explores how a PAC-proven decision tree learning algorithm fares in comparison with two variants of the normal top-down induction of decision trees, including a backtracking search algorithm for learning rank-minimal decision trees.
Theoretical Computer ScienceEffects of domain characteristics on instance-based learning algorithms
20 Citations2003Seishi Okamoto, Nobuhiro Yugami
This paper presents average-case analyses of instance-based learning algorithms using a variant of k-nearest neighbor classifier (k-NN) and presents the behavioral implications of the analyses by presenting the effects of domain characteristics on the expected accuracy of k -NN and on the optimal value of k for artificial domains.
Lecture notes in computer scienceMethods for Rule Conflict Resolution
20 Citations2004Tony Lindgren
A survey of methods used to solve rule conflicts and a novel method called Recursive Induction is introduced, which shows that Recursion outperforms all previously used methods.
Lecture notes in computer scienceOnline Learning of Linear Classifiers
15 Citations2003Jyrki Kivinen
The proof for generalizing the perceptron convergence theorem for the p-norm perceptron and the non-separable case is given and it is shown how regularization can make an online algorithm more robust against target movement.
…