Metalearning: a survey of trends and technologies
Artificial Intelligence ReviewPublished 19 July 2013Open access
Christiane Lemke, Marcin Budka, Bogdan Gabryś
Citations374
SJR quartileQ1
SJR score3.01
SNIP5.11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
An all-encompassing overview of the research directions pursued under the umbrella of metalearning is given, reconciling different definitions given in scientific literature, listing the choices involved when designing aMetalearning system and identifying some of the future research challenges in this domain are identified.
Abstract
Metalearning attracted considerable interest in the machine learning community in the last years. Yet, some disagreement remains on what does or what does not constitute a metalearning problem and in which contexts the term is used in. This survey aims at giving an all-encompassing overview of the research directions pursued under the umbrella of metalearning, reconciling different definitions given in scientific literature, listing the choices involved when designing a metalearning system and identifying some of the future research challenges in this domain.
Keywords
Computer Science
IEEE Transactions on Knowledge and Data EngineeringA Survey on Transfer Learning
23,164 Citations2009Sinno Jialin Pan, Qiang Yang
The relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift are discussed.
Journal of Computer and System SciencesA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
20,320 Citations1997Yoav Freund, Robert E. Schapire
Choice Reviews OnlineNeural networks for pattern recognition
18,687 Citations1994
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Neural Networks for Pattern Recognition
12,198 Citations1995Chris Bishop
Lecture notes in computer scienceEnsemble Methods in Machine Learning
7,862 Citations2000Thomas G. Dietterich
Neural NetworksStacked generalization
7,478 Citations1992David H. Wolpert
The conclusion is that for almost any real-world generalization problem one should use some version of stacked generalization to minimize the generalization error rate.
Mining high-speed data streams
2,180 Citations2000Pedro Domingos, Geoff Hulten
This paper describes and evaluates VFDT, an anytime system that builds decision trees using constant memory and constant time per example, and applies it to mining the continuous stream of Web access data from the whole University of Washington main campus.
ACM Computing SurveysData fusion
1,829 Citations2009Jens Bleiholder, Felix Naumann
This article places data fusion into the greater context of data integration, precisely defines the goals of data fusion, namely, complete, concise, and consistent data, and highlights the challenges of data Fusion.
Advances in computersThe Algorithm Selection Problem
1,128 Citations1976John R. Rice
This chapter starts with a discussion on abstract models: the basic model and associated problems, the model with selection based on features, and themodel with variable performance criteria, to explore the applicability of the approximation theory to the algorithm selection problem.
Artificial Intelligence ReviewA Perspective View and Survey of Meta-Learning
1,114 Citations2002Ricardo Vilalta, Youssef Drissi
This paper provides its own perspective view in which the goal is to build self-adaptive learners that improve their bias dynamically through experience by accumulating meta-knowledge, and provides a survey of meta-learning as reported by the machine-learning literature.
Journal of Machine Learning ResearchLearning Multiple Tasks with Kernel Methods
836 Citations2005Theodoros Evgeniou, Charles A. Micchelli +1 more
The experiments show that learning multiple related tasks simultaneously using the proposed approach can significantly outperform standard single-task learning particularly when there are many related tasks but few data per task.
British Journal of Educational PsychologyTHE ROLE OF METALEARNING IN STUDY PROCESSES
548 Citations1985John Biggs
ACM Computing SurveysCross-disciplinary perspectives on meta-learning for algorithm selection
544 Citations2009Kate Smith‐Miles
The generalization of meta-learning concepts to algorithms focused on tasks including sorting, forecasting, constraint satisfaction, and optimization, and the extension of these ideas to bioinformatics, cryptography, and other fields are discussed.
Metalearning - Applications to Data Mining
464 Citations2009Pavel Brazdil, Christophe Giraud-Carrier +2 more
Machine LearningRanking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results
404 Citations2003Pavel Brazdil, Carlos Soares +1 more
A meta-learning method that uses a k-Nearest Neighbor algorithm to identify the datasets that are most similar to the one at hand and leads to significantly better rankings than the baseline ranking method.
Procedia Computer ScienceStream Processing of Healthcare Sensor Data: Studying User Traces to Identify Challenges from a Big Data Perspective
362 Citations2015Rudyar Cortés, Xavier Bonnaire +2 more
A detailed analysis of traces gathered from a well-known healthcare sport-oriented application is provided in order to illustrate the conclusions from a big data perspective.
Meta-Learning by Landmarking Various Learning Algorithms
302 Citations2000Bernhard Pfahringer, Hilan Bensusan +1 more
Experiments show that landmarking selects, with moderate but reasonable level of success, the best performing of a set of learning algorithms.
MIS QuarterlyMetaFraud: A Meta-Learning Framework for Detecting Financial Fraud1
294 Citations2012Ahmed Abbasi, Albrecht +2 more
The MetaFraud framework is developed, a novel meta-learning framework for enhanced financial fraud detection that generates confidence scores associated with each prediction that can facilitate unprecedented financial fraud Detection performance and serve as a useful decision-making aid.
Machine LearningCascade Generalization
274 Citations2000João Gama, Pavel Brazdil
Two related methods for merging classifiers are presented, one of which outperforms other methods for combining classifiers, like Stacked Generalization, and competes well against Boosting at statistically significant confidence levels.
NeurocomputingMeta-learning for time series forecasting and forecast combination
249 Citations2010Christiane Lemke, Bogdan Gabryś
This work identifies an extensive feature set describing both the time series and the pool of individual forecasting methods and investigates the applicability of different meta-learning approaches, showing the superiority of a ranking-based combination of methods over simple model selection approaches.
Machine LearningCombining Classifiers with Meta Decision Trees
203 Citations2003Ljupčo Todorovski, Sašo Džeroski
The paper introduces meta decision trees (MDTs), a novel method for combining multiple classifiers that instead of giving a prediction, MDT leaves specify which classifier should be used to obtain a prediction.
IEEE Transactions on Knowledge and Data EngineeringToward intelligent assistance for a data mining process: an ontology-based approach for cost-sensitive classification
193 Citations2005Abraham Bernstein, Foster Provost +1 more
A prototype intelligent discovery assistant (IDA) is presented, which provides users with systematic enumerations of valid DM processes, in order that important, potentially fruitful options are not overlooked and effective rankings of these valid processes by different criteria are facilitated to facilitate the choice of DM processes to execute.
NeurocomputingRule induction for forecasting method selection: Meta-learning the characteristics of univariate time series
176 Citations2008Xiaozhe Wang, Kate Smith‐Miles +1 more
A novel approach for selecting a forecasting method for univariate time series based on measurable data characteristics is presented that combines elements of data mining, meta-learning, clustering, classification and statistical measurement.
Lecture notes in computer scienceA Comparison of Ranking Methods for Classification Algorithm Selection
172 Citations2000Pavel Brazdil, Carlos Soares
This work investigates the problem of using past performance information to select an algorithm for a given classification problem, and presents three ranking methods for that purpose: average ranks, success rate ratios and significant wins.
The R JournalPMML: An Open Standard for Sharing Models
159 Citations2009Alex Guazzelli, M. Zeller +2 more
The PMML package exports a variety of predictive and descriptive models from R to the Predictive Model Markup Language, an XML-based language that allows for the interchange of models among different tools and environments, mostly avoiding proprietary issues and incompatibilities.
IEEE Transactions on Knowledge and Data EngineeringDomain-Driven Data Mining: Challenges and Prospects
152 Citations2010Longbing Cao
An overview of driving forces, theoretical frameworks, architectures, techniques, case studies, and open issues of domain-driven data mining shows that the findings are not actionable, and lack soft power in solving real-world complex problems.
Machine LearningMeta-learning for evolutionary parameter optimization of classifiers
149 Citations2012Matthias Reif, Faisal Shafait +1 more
NeurocomputingMeta-learning approaches to selecting time series models
140 Citations2004Ricardo B. C. Prudêncio, Teresa B. Ludermir
An original work that applies meta-learning approaches to select models for time-series forecasting and uses the NOEMON approach to rank three models used to forecast time series of the M3-Competition.
NeurocomputingCombining meta-learning and search techniques to select parameters for support vector machines
140 Citations2011Taciana A.F. Gomes, Ricardo B. C. Prudêncio +3 more
This work investigates the combination of meta-learning with two search algorithms: Particle Swarm Optimization and Tabu Search and showed that, on average, the proposed hybrid methods obtained lower error rates when compared to their components applied in isolation.
Pattern Analysis and ApplicationsAutomatic classifier selection for non-experts
139 Citations2012Matthias Reif, Faisal Shafait +3 more
This paper empirically evaluate five different categories of state-of-the-art meta-features for their suitability in predicting classification accuracies of several widely used classifiers and develops the first open source meta-learning system that is capable of accurately predicting accuraciesof target classifiers.
IEEE Computational Intelligence MagazineEvolving artificial neural network ensembles
127 Citations2008Xin Yao, Md. Monirul Islam
Machine LearningFlexible latent variable models for multi-task learning
105 Citations2008Jian Zhang, Zoubin Ghahramani +1 more
A probabilistic framework which can support a set of latent variable models for different multi-task learning scenarios and it is shown that the framework is a generalization of standard learning methods for single prediction problems and it can effectively model the shared structure among different prediction tasks.
Studies in computational intelligenceMeta-Learning in Computational Intelligence
94 Citations2011Jankowski, Norbert, Gra̧bczewski, Krzysztof +1 more
This book defines and reveals new theoretical and practical trends in meta-learning, inspiring the readers to further research in this exciting field.
Intelligent Data AnalysisNOEMON: Design, implementation and performance results of an intelligent assistant for classifier selection
88 Citations1999Alexandros Kalousis
The MiningMart Approach to Knowledge Discovery in Databases
83 Citations2004Katharina Morik, Martin Scholz
The MiningMart system presented in this chapter focuses on setting up and re-using best practice cases of preprocessing data stored in very large databases using a metadata model named M4, which allows reuse of best practice cases published on the Internet.
Bristol Research (University of Bristol)A higher-order approach to meta-learning
75 Citations2000Hilan Bensusan, Christophe Giraud-Carrier +1 more
This position paper proposes to (meta-)learn directly from induced decision trees, rather than rely on an ad hoc set of pre-computed characteristics, within the framework of the typed higher-order inductive learning framework developed.
Expert Systems with ApplicationsLoad forecasting using a multivariate meta-learning system
64 Citations2013Marin Matijaš, Johan A. K. Suykens +1 more
A meta-learning system for multivariate time-series forecasting as a general framework for load forecasting model selection and new metafeatures of fickleness, traversity, granularity and highest ACF are introduced.
Lecture notes in computer scienceFeature Selection for Meta-learning
56 Citations2001Alexandros Kalousis, Mélanie Hilario
This work discovered that the best set of discriminating attributes is different for every pair of inducers, and applied a feature selection method on the meta-learning problems, to get thebest set of attributes for each problem.
ACM SIGKDD Explorations NewsletterPostprocessing in machine learning and data mining
52 Citations2000Ivan Brůha, A. Fazel Famili
This article surveys the contents of the workshop Post-Processing in Machine Learning and data Mining: Interpretation, Visualization, Integration, and Related Topics within KDD-2000: The Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
Machine LearningInductive transfer with context-sensitive neural networks
50 Citations2008Daniel Silver, Ryan Poirier +1 more
Context-sensitive Multiple Task Learning is presented as a method of inductive transfer which uses a single output neural network and additional contextual inputs for learning multiple tasks and is shown to produce hypotheses for primary tasks that are often better than standard MTL hypotheses when learning in the presence of related and unrelated tasks.
Lecture notes in computer scienceRanking with Predictive Clustering Trees
49 Citations2002Ljupčo Todorovski, Hendrik Blockeel +1 more
This work proposes to use predictive clustering trees for ranking, as compared to existing ranking approaches which are instance-based, and also allows for an explanation of the predicted rankings.
Memetic ComputingArchitecture for development of adaptive on-line prediction models
48 Citations2009Petr Kadlec, Bogdan Gabryś
In order to demonstrate its practicality, it is shown how the issues of current soft sensor development and maintenance can be effectively dealt with by using the architecture as a construction plan for the development of adaptive soft sensing algorithms.
Lecture notes in computer scienceExperiments in Meta-level Learning with ILP
41 Citations1999Ljupčo Todorovski, Sašo Džeroski
Machine LearningGuest editor’s introduction: special issue on inductive transfer learning
39 Citations2008Daniel Silver, Kristin P. Bennett
While people are adept at inductive transfer, the authors have only begun to develop associated computational learning theory and there are few machine learning systems that exhibit knowledge transfer.
Journal of ForecastingA Meta‐learning Framework for Bankruptcy Prediction
38 Citations2011Chih‐Fong Tsai, Yu‐Feng Hsu
A meta-learning framework, which is composed of two-level classifiers for bankruptcy prediction, provides higher prediction accuracy rates and lower type I/II errors when compared with the stacked generalization classifier and other three widely developed baselines, such as neural networks, decision trees, and logistic regression.
Lecture notes in computer scienceUsing Machine Learning Techniques to Combine Forecasting Methods
34 Citations2004Ricardo B. C. Prudêncio, Teresa B. Ludermir
An original work that uses machine learning techniques to combine time series forecasts and implemented a prototype that uses a MLP network to combine two widespread methods.
Lecture notes in computer scienceTowards process-oriented tool support for knowledge discovery in databases
34 Citations1997Rüdiger Wirth, Colin Shearer +6 more
This paper develops a scaleable, extensible tool inherently based on the view of KDD as an interactive and iterative process, namely an information manager for effective retrieval of data and results, an execution server for efficient execution, and a process support interface for guiding the user through the process.
The Data Mining Advisor: Meta-learning at the Service of Practitioners
32 Citations2006Christophe Giraud-Carrier
The potential of meta-learning is reviewed in this context and the early success of a Web-based data mining assistant is reported on, which shows the potential of this approach in the context of classification.
Lecture notes in computer scienceUCI++: Improved Support for Algorithm Selection Using Datasetoids
30 Citations2009Carlos Soares
This paper proposes a method to obtain a large number of datasets which is based on a simple transformation of existing datasets, referred to as datasetoids, and test the approach on the problem of using metalearning to predict when to prune decision trees.
Educational Data MiningMeta-learning Approach for Automatic Parameter Tuning: A case of study with educational datasets.
25 Citations2012María De Mar Molina, Cristóbal Romero +2 more
A meta-learning approach for automatic parameter tuning of a well-known decision tree algorithm by using past information about algorithm executions to help decide how the default parameters should be tuned in order to increase the accuracy of the classifier when using different types of educational datasets.
Water ResearchRobust predictive modelling of water pollution using biomarker data
22 Citations2010Marcin Budka, Bogdan Gabryś +1 more
A step-by-step, systematic data analysis approach is presented, resulting in design of a purely data-driven model, able to accurately discriminate between various coastal water pollution levels, based on low quality biomarker data.
Studies in computational intelligenceUniversal Meta-Learning Architecture and Algorithms
21 Citations2011Norbert Jankowski, Krzysztof Grąbczewski
Dynamic combination of forecasts generated by diversification procedures applied to forecasting of airline cancellations
19 Citations2009Christiane Lemke, Silvia Riedel +1 more
A dynamic approach of pooling and trimming is applied to the generated forecasts for airline cancellation data, with the aim of improving the forecast of cancellation rates for bookings.
Procedia Computer ScienceRidge regression ensemble for toxicity prediction
19 Citations2010Marcin Budka, Bogdan Gabryś
This paper proposes a purely data–driven, rigorous and universal methodology of QSAR modelling, based on ensemble of relatively simple ridge regressors trained in various subspaces of the chemical space, selected using an iterative optimization procedure.
Lecture notes in computer scienceLazy Meta-Learning: Creating Customized Model Ensembles on Demand
15 Citations2012Piero P. Bonissone
This work proposes customized model ensembles on demand, inspired by Lazy Learning, which finds the most relevant models from a DB of models, using their meta-information, and creates an ensemble, which produces an output that is a weighted interpolation or extrapolation of the outputs of the models ensemble.
Lecture notes in computer scienceAn Experimental Study of the Combination of Meta-Learning with Particle Swarm Algorithms for SVM Parameter Selection
14 Citations2012Péricles Miranda, Ricardo B. C. Prudêncio +2 more
This research analysed the influence of meta-learning on convergence of the optimization algorithms, verifying that the combination of PSO techniques with ML obtained solutions with higher quality on a set of 40 classification problems.
A Characterization of Difficult Problems in Classification.
11 Citations2002Ricardo Vilalta, Youssef Drissi
A new and distinct cultivar of geranium plant named HWD Campana is characterized by its semi-double purple flowers; compact plant size; freely branching habit; and a large number of umbels per plant.
Learnt Topology Gating Artificial Neural Networks
10 Citations2008Petr Kadlec, Bogdan Gabryś
This work combines several established regression and meta-learning techniques to give a holistic regression model and presents the proposed learnt topology gating artificial neural networks (LTGANN) model in the context of a general architecture previously published by the authors.
Lirias (KU Leuven)Towards understanding learning behavior
9 Citations2006Joaquin Vanschoren, Hendrik Blockeel
A descriptive meta-learning approach is proposed, i.e. being able to thoroughly investigate and explain the reasons behind the success or failure of a learning algorithm, based on synthetic datasets, “insightful” meta-features (about data as well as algorithms) and eventually incorporating preprocessing techniques.
On the Use of Fast Subsampling Estimates for Algorithm Recommendation
8 Citations2002Johannes Fürnkranz, Johann Petrak +2 more
The main goal of the paper is to investigate the influence of various parameter choices on the meta-learning performance, in particular the size of training and test sets and the number of subsamples.
Research Padua Archive (University of Padua)Transfer Learning by Kernel Meta-Learning.
8 Citations2011Fabio Aiolli
A kernel meta-learning algorithm which, starting from a basic kernel, tries to learn chains of kernel transforms that are able to produce good kernel matrices for the source tasks, which can be applied to compute the kernel matrix for new related target tasks.
Lecture notes in computer scienceComplexity Measures for Meta-learning and Their Optimality
6 Citations2013Norbert Jankowski
Meta-learning can be seen as alternating the construction of configuration of learning machines for validation, scheduling of such tasks and the meta-knowledge collection.
DOAJ (DOAJ: Directory of Open Access Journals)METALA: a J2EE technology based framework for web mining
4 Citations2004Juan M. Hernansáez, Juan A. Botía +1 more
The most important aspects of METALA, a software tool for meta-learning that has developed to perform inductive learning in a distributed and component based fashion, are discussed.
Lirias (KU Leuven)Meta-learning from an experiment database
2 Citations2012Kurt Driessens, Gitte Vanwinckelen +1 more
The goal of the project was to analyze a Machine Learning Experiment database, using standard SQL queries and data mining tools with the goals of giving the students some practice with applying the machine learning techniques on a real problem.