The Data Mining Advisor: Meta-learning at the Service of Practitioners
Published 22 March 2006
Christophe Giraud-Carrier
Citations32
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The potential of meta-learning is reviewed in this context and the early success of a Web-based data mining assistant is reported on, which shows the potential of this approach in the context of classification.
Abstract
In order to make machine learning algorithms more usable, our community must be able to design robust systems that offer support to practitioners. In the context of classification, this amounts to developing assistants, which deal with the increasing number of models and techniques, and give advice dynamically on such issues as model selection and method combination. This paper briefly reviews the potential of meta-learning in this context and reports on the early success of a Web-based data mining assistant.
Keywords
Computer Science
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
Programs for Machine Learning
5,794 Citations1994Steven L. Salzberg, Alberto M. Segre
In his new book, C4.5: Programs for Machine Learning, Quinlan has put together a definitive, much needed description of his complete system, including the latest developments, which will be a welcome addition to the library of many researchers and students.
Management ScienceA Procedure for Ranking Efficient Units in Data Envelopment Analysis
3,673 Citations1993Per Kragh Andersen, Niels Christian Petersen
TechnometricsMachine Learning, Neural and Statistical Classification
2,188 Citations1995Bill Fulkerson, D. Michie +2 more
Machine LearningVery Simple Classification Rules Perform Well on Most Commonly Used Datasets
1,798 Citations1993Robert C. Holte
On most datasets studied, the best of very simple rules that classify examples on the basis of a single attribute is as accurate as the rules induced by the majority of machine learning systems.
RePEc: Research Papers in EconomicsNo Free Lunch Theorems for Search
924 Citations1995David H. Wolpert, William G. Macready
It is shown that all algorithms that search for an extremum of a cost function perform exactly the same, when averaged over all possible cost functions, which allows for mathematical benchmarks for assessing a particular search algorithm's performance.
Towards parameter-free data mining
568 Citations2004Eamonn Keogh, Stefano Lonardi +1 more
This work shows that recent results in bioinformatics and computational theory hold great promise for a parameter-free data-mining paradigm, and shows that this approach is competitive or superior to the state-of-the-art approaches in anomaly/interestingness detection, classification, and clustering with empirical tests on time series/DNA/text/video datasets.
The Supervised Learning No-Free-Lunch Theorems
461 Citations2002David H. Wolpert
This paper reviews the supervised learning versions of the no-free-lunch theorems in a simplified form, and discusses the significance of those theoresms, and their relation to other aspects of supervised learning.
Machine LearningRanking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results
404 Citations2003Pavel Brazdil, Carlos Soares +1 more
A meta-learning method that uses a k-Nearest Neighbor algorithm to identify the datasets that are most similar to the one at hand and leads to significantly better rankings than the baseline ranking method.
Elsevier eBooksA Conservation Law for Generalization Performance
341 Citations1994Cullen Schaffer
A proof of a basic mathematical result stating that positive performance in some learning situations must be offset by an equal degree of negative performance in others is presented.
Meta-Learning by Landmarking Various Learning Algorithms
302 Citations2000Bernhard Pfahringer, Hilan Bensusan +1 more
Experiments show that landmarking selects, with moderate but reasonable level of success, the best performing of a set of learning algorithms.
Lecture notes in computer scienceA Comparison of Ranking Methods for Classification Algorithm Selection
172 Citations2000Pavel Brazdil, Carlos Soares
This work investigates the problem of using past performance information to select an algorithm for a given classification problem, and presents three ranking methods for that purpose: average ranks, success rate ratios and significant wins.
IEEE Transactions on Pattern Analysis and Machine IntelligenceMeta analysis of classification algorithms for pattern recognition
120 Citations1999So Young Sohn
A statistical meta-model is developed which compares the classification performances of several algorithms in terms of data characteristics and is expected to aid decision making processes of finding the best classification tool in the sense of providing the minimum classification error among alternatives.
Lecture notes in computer scienceImproved Dataset Characterisation for Meta-learning
115 Citations2002Yonghong Peng, Peter Flach +2 more
New measures, based on the induced decision tree, to characterise datasets for meta-learning in order to select appropriate learning algorithms, and their effectiveness is illustrated through extensive experiments.
Lecture notes in computer scienceAST: Support for Algorithm Selection with a CBR Approach
102 Citations1999Guido Lindner, Rudi Studer
The Algorithm Selection Tool (AST) is described that is one component in a general framework for defining user support for Data Mining algorithms in the field of Knowledge Discovery in Databases (KDD).
Lecture notes in computer scienceEstimating the Predictive Accuracy of a Classifier
91 Citations2001Hilan Bensusan, Alexandros Kalousis
It is shown that it is possible to estimate classifier performance with a high degree of confidence and gain knowledge about the classifier through the regression models generated and exploit the results of the models to predict the ranking of the inducers.
International Journal on Cyber Situational AwarenessA Public-Private-Partnership Model for National Cyber Situational Awareness
78 Citations2016Tímea Páhi, Florian Skopik
This paper describes an idealized meta-learning architecture comprising a variety of relevant component techniques and shows how metalearning has already been identified as an important component in real-world applications.
Lecture notes in computer scienceDiscovering Task Neighbourhoods through Landmark Learning Performances
75 Citations2000Hilan Bensusan, Christophe Giraud-Carrier
Landmarking is a novel and promising metalearning approach to model selection that uses accuracy estimates from simple and efficient learners to describe tasks and subsequently construct meta-classifiers that predict which one of a set of more elaborate learning algorithms is appropriate for a given problem.
Bristol Research (University of Bristol)A higher-order approach to meta-learning
75 Citations2000Hilan Bensusan, Christophe Giraud-Carrier +1 more
This position paper proposes to (meta-)learn directly from induced decision trees, rather than rely on an ad hoc set of pre-computed characteristics, within the framework of the typed higher-order inductive learning framework developed.
Model selection via meta-learning: a comparative study
67 Citations2002Alexandros Kalousis, Mélanie Hilario
This work extends and refine the set of data characteristics and uses a wider range of base-level inducers and a much larger collection of datasets to create the meta-models and shows that decision trees and boosted decision trees models enhance the perfomance of the system.
An Evaluation of Landmarking Variants
63 Citations2001Johannes Fürnkranz, Johann Petrak
This paper investigates relative landmarking, which tries to exploit the relative order of the landmark measures instead of their absolute value, and proposes to use subsampling estimates as a different way for efficiently obtaining landmarks.
Using a Data Metric for Preprocessing Advice for Data Mining Applications.
60 Citations1998Robert Engels, C. Theusinger
This paper describes research that is performed in the course of a project where a methodology for providing user support for KDD processes plays a central role and gives some insights in the metadata the authors calculate from a dataset as part of the method for user support.
Beyond predictive accuracy : what?
31 Citations1998Christophe Giraud-Carrier
A number of criteria that could, and often ought to, be considered when learning about model selection are presented and the impact they have on meta-level approaches to model selection is discussed.
Lecture notes in computer scienceFusion of Meta-knowledge and Meta-data for Case-Based Model Selection
27 Citations2001Mélanie Hilario, Alexandros Kalousis
A case-based system for model selection which combines knowledge and data in selecting a (set of) algorithm(s) to recommend for a given task and can be used not only as a predictive tool but also as an exploratory tool for gaining further insight into previously tested algorithms and datasets.
Lecture notes in computer scienceModel Class Selection and Construction: Beyond the Procrustean Approach to Machine Learning Applications
21 Citations2001Maarten van Someren
Machine Learning was primarily inspired by human learning and scientists tried to build systems that reproduce forms of human learning but currently the methods that were discovered have been elaborated and are applied to tasks that are not performed by humans at all.
Predicting Algorithm Accuracy with a Small Set of Effective Meta-Features
12 Citations2008Jun‐Won Lee, Christophe Giraud-Carrier
4 meta-features whose values are directly relevant to certain ranges of predictive accuracy for 7 learning algorithms on 135 UCI datasets are found using visual analysis and computational complexity considerations.
Odd bites into bananas don''t make you blind: learning about simplicity and attribute addition
10 Citations1998Hilan Bensusan
A meta-learning technique that uses unpruned decision trees and information from decision tree construction to describe the learning tasks is described and results suggest that induction on the connection between problems and simplicity and representation biases improves learning performance.
NASA Technical Reports Server (NASA)Any Two Learning Algorithms Are (Almost) Exactly Identical
8 Citations2000David H. Wolpert
This paper shows that if one is provided with a loss function, it can be used in a natural way to specify a distance measure quantifying the similarity of any two supervised learning algorithms, even non-parametric algorithms, indicating that any two learning algorithms are almost exactly identical for such scenarios.