On Data and Algorithms: Understanding Inductive Performance
Machine LearningPublished 13 February 2004Open access
Alexandros Kalousis, João Gama, Mélanie Hilario
Citations81
SJR quartileQ1
SJR score1.15
SNIP2.14
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper addresses two symmetrical issues, the discovery of similarities among classification algorithms, and among datasets, on the basis of error measures, which are used to discover similarities between learners, and both of them to discovering similarities between datasets.
Abstract
Abstract. In this paper we address two symmetrical issues, the discov-ery of similarities among classification algorithms, and among datasets. Both on the basis of error measures, which we use to define the error cor-relation between two algorithms, and determine the relative performance of a list of algorithms. We use the first to discover similarities between learners, and both of them to discover similarities between datasets. The latter sketch maps on the dataset space. Regions within each map exhibit specific patterns of error correlation or relative performance. To acquire an understanding of the factors determining these regions we describe them using simple characteristics of the datasets. Descriptions of each region are given in terms of the distributions of dataset characteristics within it. 1
Keywords
Computer Science
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
Pattern classification and scene analysis
12,643 Citations1973Richard O. Duda, Peter E. Hart
Neural NetworksStacked generalization
7,478 Citations1992David H. Wolpert
The conclusion is that for almost any real-world generalization problem one should use some version of stacked generalization to minimize the generalization error rate.
Programs for Machine Learning
5,794 Citations1994Steven L. Salzberg, Alberto M. Segre
In his new book, C4.5: Programs for Machine Learning, Quinlan has put together a definitive, much needed description of his complete system, including the latest developments, which will be a welcome addition to the library of many researchers and students.
ePrints Soton (University of Southampton)An Introduction to Support Vector Machines
4,120 Citations2000Nello Cristianini, John Shawe‐Taylor
Elsevier eBooksFast Effective Rule Induction
3,767 Citations1995William W. Cohen
This paper evaluates the recently-proposed rule learning algorithm IREP on a large and diverse collection of benchmark problems, and proposes a number of modifications resulting in an algorithm RIPPERk that is very competitive with C4.5 and C 4.5rules with respect to error rates, but much more efficient on large samples.
Journal of the Royal Statistical Society Series A (General)Pattern Classification and Scene Analysis.
3,240 Citations1974Mairi Clarke, Richard O. Duda +1 more
TechnometricsMachine Learning, Neural and Statistical Classification
2,188 Citations1995Bill Fulkerson, D. Michie +2 more
Medical Entomology and ZoologyAnalysis of Symbolic Data: Exploratory Methods for Extracting Statistical Information from Complex Data
557 Citations2000Hans Hermann Bock, Edwin Diday
This work focuses on Symbolic Data Analysis and the SODAS Project: Purpose, History, Perspective, and Symbolic Objects, where H.H. Bock and E. Diday focused on the former and the latter dealt with the latter.
Machine LearningRanking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results
404 Citations2003Pavel Brazdil, Carlos Soares +1 more
A meta-learning method that uses a k-Nearest Neighbor algorithm to identify the datasets that are most similar to the one at hand and leads to significantly better rankings than the baseline ranking method.
Meta-Learning by Landmarking Various Learning Algorithms
302 Citations2000Bernhard Pfahringer, Hilan Bensusan +1 more
Experiments show that landmarking selects, with moderate but reasonable level of success, the best performing of a set of learning algorithms.
Machine LearningError Reduction through Learning Multiple Descriptions
249 Citations1994Kamal Ali, Michael J. Pazzani
It is empirically shown that it is possible to learn descriptions that make less correlated errors in domains in which many ties in the search evaluation measure (e.g. information gain) are experienced during learning.
Elsevier eBooksGeneralizing from Case Studies: A Case Study
214 Citations1992David W. Aha
An empirical method for generalizing results from case studies and an example application is described, which yields rules describing when some algorithms significantly outperform others on some dependent measures.
Machine LearningError reduction through learning multiple descriptions
162 Citations1996Kamal Ali, Michael J. Pazzani
It is empirically shown that it is possible to learn descriptions that make less correlated errors in domains in which many ties in the search evaluation measure (e.g. information gain) are experienced during learning.
Lecture notes in computer scienceCharacterization of classification algorithms
117 Citations1995João Gama, Pavel Brazdil
This paper is concerned with the problem of characterization of classification algorithms and involves generation of different kinds of models, which include regression and rule models, piecewise linear models (model trees) and instance based models.
Lecture notes in computer scienceImproved Dataset Characterisation for Meta-learning
115 Citations2002Yonghong Peng, Peter Flach +2 more
New measures, based on the induced decision tree, to characterise datasets for meta-learning in order to select appropriate learning algorithms, and their effectiveness is illustrated through extensive experiments.
International Joint Conference on Artificial IntelligenceAverage-case analysis of a nearest neighbor algorthim
89 Citations1993Pat Langley, Wayne Iba
An average-case analysis of the nearest neighbor algorithm, a simple induction method that has been studied by many researchers, is presented by presenting predicted learning curves for artificial domains, and experimental results on these domains are given as a check on the reasoning.
Intelligent Data AnalysisNOEMON: Design, implementation and performance results of an intelligent assistant for classifier selection
88 Citations1999Alexandros Kalousis
Lecture notes in computer scienceZoomed Ranking: Selection of Classification Algorithms Based on Relevant Performance Information
84 Citations2000Carlos Soares, Pavel Brazdil
The adjusted ratio of ratios ranking method takes into account not only accuracy but also the time performance of the candidate algorithms, and indicates that on average better results are obtained with zooming than without it.
Model selection via meta-learning: a comparative study
67 Citations2002Alexandros Kalousis, Mélanie Hilario
This work extends and refine the set of data characteristics and uses a wider range of base-level inducers and a much larger collection of datasets to create the meta-models and shows that decision trees and boosted decision trees models enhance the perfomance of the system.
International Journal of Artificial Intelligence ToolsMODEL SELECTION VIA META-LEARNING: A COMPARATIVE STUDY
58 Citations2001Alexandros Kalousis, HILARIO MELANIE
The selection of an appropriate inducer is crucial for performing effective classification and this work presented a system called NOEMON which relied on a mapping between dataset characteri...
Lecture notes in computer scienceFeature Selection for Meta-learning
56 Citations2001Alexandros Kalousis, Mélanie Hilario
This work discovered that the best set of discriminating attributes is different for every pair of inducers, and applied a feature selection method on the meta-learning problems, to get thebest set of attributes for each problem.
Lecture notes in computer scienceRanking with Predictive Clustering Trees
49 Citations2002Ljupčo Todorovski, Hendrik Blockeel +1 more
This work proposes to use predictive clustering trees for ranking, as compared to existing ranking approaches which are instance-based, and also allows for an explanation of the predicted rankings.
Lecture notes in computer scienceExperiments in Meta-level Learning with ILP
41 Citations1999Ljupčo Todorovski, Sašo Džeroski
Tractable Average-Case Analysis of Naive Bayesian Classifiers
25 Citations1999Pat Langley, Stephanie Sage
An average case anal ysis of the naive Bayesian classi er a sim ple induction algorithm that performs well in many domains and makes very accurate predictions despite its use of approximations is presented.
Intelligent Data AnalysisNOEMON: Design, implementation and performance results of an intelligent assistant for classifier selection☆
19 Citations1999Alexandros Kalousis, Theoharis Theoharis
An intelligent assistant that is able to reduce the effort of mastering a lot of background information on the dataset, the models and the algorithms in question by inducing helpful suggestions from background information is presented.
Scholarworks (University of Massachusetts Amherst)Recursive automatic algorithm selection for inductive learning
13 Citations1995Carla E. Brodley
An approach to overcoming this problem by applying knowledge about the biases of a set of learning algorithms to conduct a recursive automatic algorithm search, which permits classifiers learned by the available algorithms to be mixed in a recursive tree-structured hybrid, thereby allowing different subproblems of the learning task to be learned by different algorithms.
Intelligent Data AnalysisLinear tree
11 Citations1999João Gama, Pavel Brazdil
This paper presents system Ltree, a probabilistic tree combining a decision tree with a linear discriminant by means of constructive induction, able to define decision surfaces both orthogonal and oblique to the axes defined by the attributes of the input space.
Lecture notes in computer scienceAverage-Case Analysis of Classification Algorithms for Boolean Functions and Decision Trees
4 Citations2000Tobias Scheffer
An average-case analysis of the generalization error rate of classification algorithms with finite model classes and shows that the analysis can predict the behavior of decision tree algorithms fairly accurately even if the error histogram is estimated from a sample.