Machine learning: a review of classification and combining techniques
Artificial Intelligence ReviewPublished 1 November 2006
Sotiris Kotsiantis, Ioannis D. Zaharakis, Panagiotis Pintelas
Citations1,483
SJR quartileQ1
SJR score3.01
SNIP5.11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Various classification algorithms and the recent attempt for improving classification accuracy—ensembles of classifiers are described.
Abstract
Supervised classification is one of the tasks most frequently carried out by so-called Intelligent Systems. Thus, a large number of techniques have been developed based on Artificial Intelligence (Logic-based techniques, Perceptron-based techniques) and Statistics (Bayesian Networks, Instance-based techniques). The goal of supervised learning is to build a concise model of the distribution of class labels in terms of predictor features. The resulting classifier is then used to assign class labels to the testing instances where the values of the predictor features are known, but the value of the class label is unknown. This paper describes various classification algorithms and the recent attempt for improving classification accuracy—ensembles of classifiers.
Keywords
Computer Science
Elsevier eBooksData Mining: Practical Machine Learning Tools and Techniques
25,718 Citations2011Ian H. Witten, Eibe Frank
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
Journal of Computer and System SciencesA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
20,320 Citations1997Yoav Freund, Robert E. Schapire
Data Mining and Knowledge DiscoveryA Tutorial on Support Vector Machines for Pattern Recognition
16,433 Citations1998Christopher J. C. Burges
There are several arguments which support the observed high accuracy of SVMs, which are reviewed and numerous examples and proofs of most of the key theorems are given.
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
ACM Computing SurveysData clustering
13,184 Citations1999Anil K. Jain, M. Narasimha Murty +1 more
An overview of pattern clustering methods from a statistical pattern recognition perspective is presented, with a goal of providing useful advice and references to fundamental concepts accessible to the broad community of clustering practitioners.
Proceedings of the 24th international conference on Machine learning
11,727 Citations2007
This is an index to the papers that appear in the Proceedings of the 29th International Conference on Machine Learning (ICML-12).
IEEE Transactions on Pattern Analysis and Machine IntelligenceThe random subspace method for constructing decision forests
6,737 Citations1998Tin Kam Ho
A method to construct a decision tree based classifier is proposed that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity.
An introduction to variable and feature selection
6,577 Citations2009Isabelle Guyon
Machine LearningBayesian Network Classifiers
4,738 Citations1997Nir Friedman, Dan Geiger +1 more
Tree Augmented Naive Bayes (TAN) is single out, which outperforms naive Bayes, yet at the same time maintains the computational simplicity and robustness that characterize naive Baye.
ePrints Soton (University of Southampton)An Introduction to Support Vector Machines
4,120 Citations2000Nello Cristianini, John Shawe‐Taylor
Elsevier eBooksFast Effective Rule Induction
3,767 Citations1995William W. Cohen
This paper evaluates the recently-proposed rule learning algorithm IREP on a large and diverse collection of benchmark problems, and proposes a number of modifications resulting in an algorithm RIPPERk that is very competitive with C4.5 and C 4.5rules with respect to error rates, but much more efficient on large samples.
Neural ComputationApproximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
3,654 Citations1998Thomas G. Dietterich
This article reviews five approximate statistical tests for determining whether one learning algorithm outperforms another on a particular learning task and measures the power (ability to detect algorithm differences when they do exist) of these tests.
Artificial Intelligence ReviewA Survey of Outlier Detection Methodologies
3,349 Citations2004Joanna Hodge, Jim Austin
Intelligent Data AnalysisThe class imbalance problem: A systematic study1
3,298 Citations2002Nathalie Japkowicz, Shaju Stephen
The assumption that the class imbalance problem does not only affect decision tree systems but also affects other classification systems such as Neural Networks and Support Vector Machines is investigated.
Machine LearningLearning Bayesian Networks: The Combination of Knowledge and Statistical Data
3,242 Citations1995David Heckerman, Dan Geiger +1 more
A methodology for assessing informative priors needed for learning Bayesian networks from a combination of prior knowledge and statistical data is developed and how to compute the relative posterior probabilities of network structures given data is shown.
Machine LearningOn the Optimality of the Simple Bayesian Classifier under Zero-One Loss
3,072 Citations1997Pedro Domingos, Michael J. Pazzani
The Bayesian classifier is shown to be optimal for learning conjunctions and disjunctions, even though they violate the independence assumption, and will often outperform more powerful classifiers for common training set sizes and numbers of attributes, even if its bias is a priori much less appropriate to the domain.
Journal of Artificial Intelligence ResearchPopular Ensemble Methods: An Empirical Study
2,972 Citations1999David W. Opitz, Richard Maclin
This work suggests that most of the gain in an ensemble's performance comes in the first few classifiers combined; however, relatively large gains can be seen up to 25 classifiers when Boosting decision trees.
Machine LearningAn Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization
2,950 Citations2000Thomas G. Dietterich
The experiments show that in situations with little or no classification noise, randomization is competitive with (and perhaps slightly superior to) bagging but not as accurate as boosting, and sometimes better than randomization.
Machine LearningAn Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants
2,644 Citations1999Eric Bauer, Ron Kohavi
It is found that Bagging improves when probabilistic estimates in conjunction with no-pruning are used, as well as when the data was backfit, and that Arc-x4 behaves differently than AdaBoost if reweighting is used instead of resampling, indicating a fundamental difference.
Introduction to Bayesian Networks
2,585 Citations2019
Learning in Graphical Models
2,416 Citations1998Michael I. Jordan
Information and ComputationThe Weighted Majority Algorithm
2,022 Citations1994N. Littlestone, Manfred K. Warmuth
Lecture notes in computer scienceKNN Model-Based Approach in Classification
1,984 Citations2003Gongde Guo, Hui Wang +3 more
This paper proposes a novel kNN type method for classification that reduces the dependency on k, makes classification faster, and compares well with C5.0 and kNN in terms of classification accuracy.
Efficient Feature Selection via Analysis of Relevance and Redundancy
1,977 Citations2004Lei Yu, Huan Liu
It is shown that feature relevance alone is insufficient for efficient feature selection of high-dimensional data, and a new framework is introduced that decouples relevance analysis and redundancy analysis.
IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews)Neural networks for classification: a survey
1,659 Citations2000Gongbo Zhang
The issues of posterior probability estimation, the link between neural and conventional classifiers, learning and generalization tradeoff in classification, the feature variable selection, as well as the effect of misclassification costs are examined.
New Generation ComputingInverse entailment and progol
1,408 Citations1995Stephen Muggleton
Mode-Directed Inverse Entailment (MDIE) is introduced as a generalisation and enhancement of previous approaches for inverting deduction and an implementation of MDIE in the Progol system is described.
Research Commons (The University of Waikato)Generating Accurate Rule Sets Without Global Optimization
1,284 Citations1998Eibe Frank, Ian H. Witten
This paper presents an algorithm for inferring rules by repeatedly generating partial decision trees, thus combining the two major paradigms for rule generation—creating rules from decision trees and the separate-and-conquer rule-learning technique.
National Conference on Artificial IntelligenceBagging, boosting, and C4.S
1,262 Citations1996J. R. Quinlan
Machine LearningReduction Techniques for Instance-Based Learning Algorithms
1,188 Citations2000D.R. Wilson, Tony Martinez
Of those algorithms that provide substantial storage reduction, the DROP algorithms have the highest average generalization accuracy in these experiments, especially in the presence of uniform class noise.
Machine LearningA Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms
1,132 Citations2000Tjen-Sien Lim, Wei‐Yin Loh +1 more
Among decision tree algorithms with univariate splits, C4.5, IND-CART, and QUEST have the best combinations of error rate and speed, but C 4.5 tends to produce trees with twice as many leaves as those fromIND-Cart and QUEST.
Artificial Intelligence ReviewA Perspective View and Survey of Meta-Learning
1,114 Citations2002Ricardo Vilalta, Youssef Drissi
This paper provides its own perspective view in which the goal is to build self-adaptive learners that improve their bias dynamically through experience by accumulating meta-knowledge, and provides a survey of meta-learning as reported by the machine-learning literature.
Large margin classification using the perceptron algorithm
1,099 Citations1998Yoav Freund, Robert E. Schapire
Data Mining and Knowledge DiscoveryAutomatic Construction of Decision Trees from Data: A Multi-Disciplinary Survey
971 Citations1998Sreerama K. Murthy
This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art.
Applied Artificial IntelligenceAn analysis of four missing data treatment methods for supervised learning
889 Citations2003Gustavo Batista, Maria Carolina Monard
This analysis indicates that missing data imputation based on the k-nearest neighbor algorithm can outperform the internal methods used by C4.5 and CN2 to treat missing data, and can also outperforms the mean or mode imputation method, which is a method broadly used to treatMissing values.
Machine LearningBeing Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks
816 Citations2003Nir Friedman, Daphne Koller
This paper shows how to efficiently compute a sum over the exponential number of networks that are consistent with a fixed order over network variables, and uses this result as the basis for an algorithm that approximates the Bayesian posterior of a feature.
Artificial IntelligenceLearning Bayesian networks from data: An information-theory based approach
803 Citations2002Jie Cheng, Russell Greiner +3 more
Algorithms that use an information-theoretic analysis to learn Bayesian network structures from data, requiring only polynomial numbers of conditional independence tests in typical cases are provided.
ACM Computing SurveysComplexity and expressive power of logic programming
753 Citations2001Evgeny Dantsin, Thomas Eiter +2 more
The MIT Press eBooksAdvances in Kernel Methods
748 Citations1998
This dissertation aims to provide a history of web exceptionalism from 1989 to 2002, a period chosen in order to explore its roots as well as specific cases up to and including the year in which descriptions of “Web 2.0” began to circulate.
Artificial Intelligence ReviewA Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms
718 Citations1997Dietrich Wettschereck, David W. Aha +1 more
A class of weight-setting methods for lazy learning algorithms which use performance feedback to assign weight settings demonstrated three advantages over other methods: they require less pre-processing, perform better in the presence of interacting features, and generally require less training data to learn good settings.
Machine LearningMultiBoosting: A Technique for Combining Boosting and Wagging
699 Citations2000Geoffrey I. Webb
MultiBoosting is an extension to the highly successful AdaBoost technique for forming decision committees that is able to harness both AdaBoost's high bias and variance reduction with wagging's superior variance reduction.
Controlling the Sensitivity of Support Vector Machines
691 Citations1999K Veropoulos, I C G Campbell +1 more
Two schemes for adjusting the sensitivity and speciicity of Support Vector Machines and the description of their performance using receiver operating characteristic (ROC) curves are discussed and their use on real-life medical diagnostic tasks is illustrated.
Machine LearningOn the Learnability and Design of Output Codes for Multiclass Problems
660 Citations2002Koby Crammer, Yoram Singer
This paper discusses for the first time the problem of designing output codes for multiclass problems, and gives a time and space efficient algorithm for solving the quadratic program.
International Conference on Artificial Intelligence and StatisticsClasses of kernels for machine learning: a statistics perspective
633 Citations2002Marc G. Genton
The spectral representation of the various classes of kernels is described and a discussion on the characterization of nonlinear maps that reduce nonstationary kernels to either stationarity or local stationarity is discussed.
Machine LearningLarge Margin Classification Using the Perceptron Algorithm
594 Citations1999Yoav Freund, Robert E. Schapire
A new algorithm for linear classification which combines Rosenblatt's perceptron algorithm with Helmbold and Warmuth's leave-one-out method is introduced, which is much simpler to implement, and much more efficient in terms of computation time.
Genetic Algorithms: Principles and Perspectives: A Guide to GA Theory
562 Citations2002Colin R. Reeves, Jonathan E. Rowe
This chapter discusses GAs as Markov processes as well as the Dynamical Systems Model, which helps clarify the role of language in the development of GA performance.
Journal of Artificial Intelligence ResearchIssues in Stacked Generalization
554 Citations1999K. M. Ting, I. H. Witten
This paper addresses two crucial issues which have been considered to be a 'black art' in classification tasks ever since the introduction of stacked generalization: the type of generalizer that is suitable to derive the higher-level model, and the kind of attributes that should be used as its input.
Data Mining and Knowledge DiscoveryAdvances in Instance Selection for Instance-Based Learning Algorithms
530 Citations2002Henry Brighton, Chris Mellish
This work introduces an algorithm that rivals the most successful existing algorithm for consistency and discusses the possibility of mechanisms that provide insights into the structure of class definitions that could be useful for the data miner.
Artificial Intelligence ReviewSeparate-and-Conquer Rule Learning
523 Citations1999Johannes Fürnkranz
This paper is a survey of inductive rule learning algorithms that use a separate-and-conquer strategy and analyzes them along three different dimensions, namely their search, language and overfitting avoidance biases.
Relational Data Mining
505 Citations2006Sašo Džeroski
Applied Artificial IntelligenceData preparation for data mining
501 Citations2003Shichao Zhang, Chengqi Zhang +1 more
The importance of data preparation in data analysis is shown, some research achievements in the area of data preparedness are introduced, and some future directions of research and development are suggested.
Érudit documents and data repository (Érudit Consortium, University of Montreal)Inference for the Generalization Error
485 Citations1999Yoshua Bengio, Claude Nadeau
Machine LearningRanking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results
404 Citations2003Pavel Brazdil, Carlos Soares +1 more
A meta-learning method that uses a k-Nearest Neighbor algorithm to identify the datasets that are most similar to the one at hand and leads to significantly better rankings than the baseline ranking method.
Operations research, computer science. Interface seriesGenetic Algorithms—Principles and Perspectives
375 Citations2002Colin R. Reeves, Jonathan E. Rowe
Machine LearningDecision Tree Induction Based on Efficient Tree Restructuring
359 Citations1997Paul E. Utgoff, Neil C. Berkman +1 more
Two approaches to decision tree induction are described, one being incremental tree induction (ITI) and the other being non-incremental tree induction using a measure of tree quality instead of test quality (DMTI), which offer new computational and classifier characteristics that lend themselves to particular applications.
Data Mining and Knowledge DiscoveryRainForest—A Framework for Fast Decision Tree Construction of Large Datasets
349 Citations2000Johannes Gehrke, Raghu Ramakrishnan +1 more
This paper presents a unifying framework called Rain Forest for classification tree construction that separates the scalability aspects of algorithms for constructing a tree from the central features that determine the quality of the tree.
The Knowledge Engineering ReviewSimplifying decision trees: A survey
336 Citations1997Leonard A. Breslow, David W. Aha
This work presents a framework that organizes the approaches to tree simplification and summarizes and critique the approaches within this framework, and discusses the application of tree induction algorithms to case retrieval in case-based reasoning systems.
Advances in Inductive Logic Programming
313 Citations1996Luc De Raedt
A state-of-the-art overview of Inductive Logic Programming is provided, based on the succesful ESPRIT basic research project no. 6020, which can be used as a thorough introduction to the field.
Machine LearningConvergence of a Generalized SMO Algorithm for SVM Classifier Design
306 Citations2002S. Sathiya Keerthi, Éric Gilbert
Convergence of a generalized version of the modified SMO algorithms given by Keerthi et al. for SVM classifier design is proved and the results are extended to modifiedSMO algorithms for solving ν-SVM classifiers problems.
IEEE Transactions on Neural NetworksAn iterative pruning algorithm for feedforward neural networks
299 Citations1997Giovanna Castellano, Anna Maria Fanelli +1 more
A new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set.
Boosting and Rocchio applied to text filtering
299 Citations1998Robert E. Schapire, Yoram Singer +1 more
This paper discusses two learning algorithms for text filtering: modified Rocchio and a boosting algorithm called AdaBoost, and shows how both algorithms can be adapted to maximize any general utility matrix that associates cost for each pair of machine prediction and correct label.
Lecture notes in computer scienceLearning Bayesian Belief Network Classifiers: Algorithms and System
287 Citations2001Jie Cheng, Russell Greiner
The results show that the proposed BN and Bayes multinet classifiers are competitive with (or superior to) the best known classifiers; and that the computational time for learning and using these classifiers is relatively small, arguing that BN-based classifiers deserve more attention in the data mining community.
Machine LearningCascade Generalization
274 Citations2000João Gama, Pavel Brazdil
Two related methods for merging classifiers are presented, one of which outperforms other methods for combining classifiers, like Stacked Generalization, and competes well against Boosting at statistically significant confidence levels.
Using Analytic QP and Sparseness to Speed Training of Support Vector Machines
266 Citations1998John Platt
An algorithm for training SVMs: Sequential Minimal Optimization, or SMO, which breaks the large QP problem into a series of smallest possible QP problems which are analytically solvable and does not require a numerical QP library.
Journal of the American Statistical AssociationOn-Line Learning in Neural Networks
265 Citations2000MTW, David Saad
A coherent picture of the state of the art in the theoretical analysis of on-line learning is presented, the first in the area, and will be welcomed by mathematicians, scientists and engineers, both in industry and academia.
Constructing diverse classifier ensembles using artificial training examples
264 Citations2003Prem Melville, Raymond J. Mooney
A new method for generating ensembles that directly constructs diverse hypotheses using additional artificially-constructed training examples is presented, which consistently achieves higher predictive accuracy than both the base classifier and bagging, and also obtains higher accuracy than boosting early in the learning curve when training data is limited.
Machine LearningLazy Learning of Bayesian Rules
249 Citations2000Zijian Zheng, Geoffrey I. Webb
This paper proposes the application of lazy learning techniques to Bayesian tree induction and presents the resulting lazy Bayesian rule learning algorithm, called LBR, which can be justified by a variant of Bayes theorem which supports a weaker conditional attribute independence assumption than is required by naive Bayes.
Studies in fuzziness and soft computingA Bayesian Approach to Causal Discovery
230 Citations2006David Heckerman, Christopher Meek +1 more
The general Bayesian approach to causal discovery is described and approximation methods for missing data and hidden variables are reviewed, and differences between the Bayesian and constraint-based methods are illustrated using artificial and real examples.
IEEE Transactions on Neural NetworksConstructive neural-network learning algorithms for pattern classification
220 Citations2000Rajesh Parekh, Jihoon Yang +1 more
Two constructive learning algorithms MPyramid-real and MTiling-real are presented that extend the pyramid and tiling algorithms, respectively, for learning real to M-ary mappings and it is proved the convergence of these algorithms and empirically demonstrate their applicability to practical pattern classification problems.
Lazy Learning
217 Citations1997David W. Aha
The survey discusses distance functions, smoothing parameters, weighting functions, local model structures, regularization of the estimates and bias, assessing predictions, handling noisy data and outliers, improving the quality of predictions by tuning fit parameters, interference between old and new data, and applications of locally weighted learning.
Instance Selection and Construction for Data Mining
216 Citations2001Huan Liu, Hiroshi Motoda
This volume serves as a comprehensive reference for graduate students, practitioners and researchers in KDD to report new developments and applications, to share hard-learned experiences in order to avoid similar pitfalls, and to shed light on the future development of instance selection.
Machine LearningCombining Classifiers with Meta Decision Trees
203 Citations2003Ljupčo Todorovski, Sašo Džeroski
The paper introduces meta decision trees (MDTs), a novel method for combining multiple classifiers that instead of giving a prediction, MDT leaves specify which classifier should be used to obtain a prediction.
Machine LearningEmpirical Support for Winnow and Weighted-Majority Algorithms: Results on a Calendar Scheduling Domain
191 Citations1997Avrim Blum
A new variant on the Winnow algorithm is created that is especially suited to conditions with string-valued classifications, and an analysis of a policy for discarding predictors in Weighted-Majority that allows it to speed up as it learns.
New Generation ComputingInduction of logic programs: FOIL and related systems
181 Citations1995J. R. Quinlan, R. M. Cameron-Jones
This paper provides an overview of the principal ideas and methods used in the current version of the FOIL system, including two recent additions.
Lecture notes in computer scienceMethods for Designing Multiple Classifier Systems
177 Citations2001Fabio Roli, Giorgio Giacinto +1 more
The problem of design of multiple classifier system is discussed, and six design methods based on the so-called "overproduce and choose" paradigm are described and compared by experiments.
Machine LearningGeneral and Efficient Multisplitting of Numerical Attributes
163 Citations1999Tapio Elomaa, Juho Rousu
The well-behavedness of an evaluation function is characterized, a property that guarantees the optimal multi-partition of an arbitrary numerical domain to be defined on boundary points.
Applied Artificial IntelligenceFeature selection for the naive bayesian classifier using decision trees
148 Citations2003Chotirat Ann Ratanamahatana, Dimitrios Gunopulos
A Selective Bayesian classifier that simply uses only those features that C4.5 would use in its decision tree when learning a small example of a training set is described, a combination of the two different natures of classifiers.
Choosing between two learning algorithms based on calibrated tests
146 Citations2003Remco Bouckaert
It is posed the case that reuse of the same data causes the effective degrees of freedom to be much lower than theoretically expected, and how to calibrate the effective degree of freedom empirically for various tests is shown.
Applied IntelligenceFERNN: An Algorithm for Fast Extraction of Rules from Neural Networks
143 Citations2000Rudy Setiono, Wee Kheng Leow
Experimental results show that the size and the predictive accuracy of the tree generated are comparable to those extracted by another method which prunes and retrains the network.
Machine LearningFeature Generation Using General Constructor Functions
142 Citations2002Shaul Markovitch, Dan Rosenstein
A generalized and flexible framework that is capable of generating features from any given set of constructor functions, and was applied to a variety of classification problems and was able to generate features that were strongly related to the underlying target concepts.
Machine LearningPruning Algorithms for Rule Learning
139 Citations1997Johannes Fürnkranz
The primary goal of this paper is to show how to solve the problems of noise in decision tree learning with two new algorithms that combine and integrate pre- and post-pruning.
Knowledge-Based SystemsHybrid decision tree
134 Citations2002Zhi‐Hua Zhou, Zhao-Qian Chen
Two incremental learning procedures designed for example-incremental learning with different storage requirements are provided, which enables HDT to deal gracefully with data sets where new data are frequently appended and a hypothesis-driven constructive induction mechanism is provided,Which enables HDt to generate compact concept descriptions.
Journal of Artificial Intelligence ResearchSearching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs
124 Citations2003Silvia Acid, Luis M. de Campos
This paper proposes a new local search method that uses a different search space, and which takes account of the concept of equivalence between network structures: restricted acyclic partially directed graphs (RPDAGs).
Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks
111 Citations1995Richard Maclin, Jude Shavlik
An approach to initializing neural networks that uses competitive learning to intelligently create networks that are originally located far from the origin of weight space, thereby potentially increasing the set of reachable local minima.
Data Mining and Knowledge DiscoveryParallel Formulations of Decision-Tree Classification Algorithms
110 Citations1999Anurag Srivastava, Eui-Hong Han +2 more
This paper describes two basic parallel formulations of classification decision tree learning algorithm based on induction and proposes a hybrid method that employs the good features of these methods.
Artificial IntelligenceInductive Logic Programming: Issues, results and the challenge of Learning Language in Logic
109 Citations1999Stephen Muggleton
Advances in ILP theory and implementation related to the challenges of LLL are already producing beneficial advances in other sequence-oriented applications of ILP, and LLL is starting to develop its own character as a sub-discipline of AI involving the confluence of computational linguistics, machine learning and logic programming.
Training neural networks: backpropagation vs. genetic algorithms
99 Citations2002M. N. H. Siddique, M. O. Tokhi
In this paper a genetic algorithm is proposed for training feedforward neural networks and its performances is investigated and the results are analyzed and compared with those obtained by the backpropagation algorithm.
IEEE Transactions on Neural NetworksFeedforward networks training speed enhancement by optimal initialization of the synaptic coefficients
95 Citations2001J.Y.F. Yam, Tommy W. S. Chow
The obtained results on five well-known benchmark problems demonstrate that the proposed method deliver consistent good results compared with other weight initialization methods.
Lecture notes in computer scienceAgent-Based Distributed Data Mining: The KDEC Scheme
93 Citations2003Matthias Klusch, Stefano Lodi +1 more
This paper is intended to argue for the potential added value of using agent technology in the domain of knowledge discovery, and proposes a novel approach to distributed data clustering based on density estimation.
The MIT Press eBooksAdaptive Nearest Neighbor Classification using Support Vector Machines
90 Citations2002Carlotta Domeniconi, Dimitrios Gunopulos
A technique that computes a locally flexible metric by means of Support Vector Machines by using the maximum margin boundary found by the SVM to determine the most discriminant direction over the query's neighborhood.
Data Mining and Knowledge DiscoveryA Unifying View on Instance Selection
89 Citations2002Thomas Reinartz
This paper defines a broader perspective on focusing tasks, chooses instance selection as one particular focusing task, and outlines the specification of concrete evaluation criteria to measure success of instance selection approaches.
Lecture notes in computer scienceOn Why Discretization Works for Naive-Bayes Classifiers
89 Citations2003Ying Yang, Geoffrey I. Webb
A theorem is proved that identifies particular conditions under which discretization will result in naive-Bayes classifiers delivering the same probability estimates as would be obtained if the correct probability density functions were employed.
Data & Knowledge EngineeringMachine learning from examples: Inductive and Lazy methods
87 Citations1998Ramón López de Mántaras, Eva Armengol
Important approaches to inductive learning methods such as propositional and relational learners, with an emphasis in Inductive Logic Programming based methods, are reported, as well as to lazy methodssuch as instance-based and case-based reasoning.
The Knowledge Engineering ReviewA review of machine learning
83 Citations1997David M. Dutton, G.V. Conroy
A general review of ML is presented, but specific detail which has been covered previously is omitted, although other relevant references are noted, and later material is commented upon.
Machine LearningOn Data and Algorithms: Understanding Inductive Performance
81 Citations2004Alexandros Kalousis, João Gama +1 more
This paper addresses two symmetrical issues, the discovery of similarities among classification algorithms, and among datasets, on the basis of error measures, which are used to discover similarities between learners, and both of them to discovering similarities between datasets.
Lecture notes in computer scienceSNNB: A Selective Neighborhood Based Naïve Bayes for Lazy Learning
78 Citations2002Zhipeng Xie, Wynne Hsu +2 more
The results of the experiments show that the proposed SNNB algorithm is efficient and it outperforms Naive Bayes, and state-of-the-art classification methods NBTree, CBA, and C4.5 in terms of accuracy.
Lecture notes in computer scienceAn Introduction to Learning Fuzzy Classifier Systems
78 Citations2000Andrea Bonarini
A class of Learning Classifier Systems that learn fuzzy rule-based models, instead of interval-based or Boolean models is presented, and a general methodology to extend reinforcement distribution algorithms usually not designed to learn fuzzy models is introduced.
Journal of Computer Science and TechnologyRule extraction: Using neural networks or for neural networks?
75 Citations2004Zhi‐Hua Zhou
The fidelity-accuracy dilemma is identified and it is argued to distinguish rule extraction using neural networks and rule extraction for neural networks according to their different goals, where fidelity and accuracy should be excluded from the rule quality evaluation framework, respectively.
…