Model Complexity and Algorithm Selection in Classification
Lecture notes in computer sciencePublished 1 January 2002
Mélanie Hilario
Citations3
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The impact of model complexity on algorithm selection is studied and it is shown how the relative performance of candidate algorithms changes drastically with the choice of complexity parameters.
Abstract
Building an effective classifer involves choosing the model class with the appropriate learning bias as well as the right level of complexity within that class. These two aspects have rarely been addressed together: typically, model class (or algorithm) selection is performed on the basis of default settings, while model instance (or complexity) selection is investigated within the confines of a single model class. We study the impact of model complexity on algorithm selection and show how the relative performance of candidate algorithms changes drastically with the choice of complexity parameters.
Keywords
Computer Science
The Nature of Statistical Learning Theory
39,279 Citations1995Vladimir Vapnik
Choice Reviews OnlineNeural networks for pattern recognition
18,687 Citations1994
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Elsevier eBooksFast Effective Rule Induction
3,767 Citations1995William W. Cohen
This paper evaluates the recently-proposed rule learning algorithm IREP on a large and diverse collection of benchmark problems, and proposes a number of modifications resulting in an algorithm RIPPERk that is very competitive with C4.5 and C 4.5rules with respect to error rates, but much more efficient on large samples.
Optimal Brain Damage
2,613 Citations1989Yann LeCun, John S. Denker +1 more
A class of practical and nearly optimal schemes for adapting the size of a neural network by using second-derivative information to make a tradeoff between network complexity and training set error is derived.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)The Cascade-Correlation learning architecture
2,401 Citations2018Scott E. Fahlman, Christian Lebière
The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the connections of the network.
TechnometricsMachine Learning, Neural and Statistical Classification
2,188 Citations1995Bill Fulkerson, D. Michie +2 more
Machine LearningA Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms
1,132 Citations2000Tjen-Sien Lim, Wei‐Yin Loh +1 more
Among decision tree algorithms with univariate splits, C4.5, IND-CART, and QUEST have the best combinations of error rate and speed, but C 4.5 tends to produce trees with twice as many leaves as those fromIND-Cart and QUEST.
Neural Information Processing SystemsSkeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment
539 Citations1988Michael C. Mozer, Paul Smolensky
The basic idea is to iteratively train the network to a certain performance criterion, compute a measure of relevance that identifies which input or hidden units are most critical to performance, and automatically trim the least relevant units.
Connection ScienceUsing Relevance to Reduce Network Size Automatically
342 Citations1989Michael C. Mozer, Paul Smolensky
This paper proposes a means of using the knowledge in a network to determine the functionality or relevance of individual units, both for the purpose of understanding the network's behavior and impinging on the structure of the network.
Machine LearningMultiple Comparisons in Induction Algorithms
243 Citations2000David Jensen, Paul R. Cohen
This work analyzes the statistical properties of MCPs and shows how failure to adjust for these properties leads to the pathologies of induction algorithms, including attribute selection errors, overfitting, and oversearching.
MLC++: a machine learning library in C++
197 Citations2002Ron Kohavi, George H. John +3 more
The problems M LC++ aims to solve, the design of MLC++, and the current functionality are discussed, including the attempt to extract commonalities of algorithms and decompose them for a unified view that is simple, coherent, and extensible.
International Journal of Pattern Recognition and Artificial IntelligenceGAL: NETWORKS THAT GROW WHEN THEY LEARN AND SHRINK WHEN THEY FORGET
94 Citations1994Ethem Alpaydın
This paper advocates the view that the network structure should not, as is usually done, be determined by trial-and-error but should be computed by the learning algorithm.
Development of multi-criteria metrics for evaluation of data mining algorithms
74 Citations1997Gholamreza Nakhaeizadeh, Alexander Schnabl
The main aim of this paper is to suggest multi-criteria based metrics that can be used as comparators for an objective evaluation of data mining algorithms (DM-algorithms), based on DEA (Data Envelopment Analysis).
Statistical Ideas for Selecting Network Architectures
56 Citations1995B. D. Ripley
The architecture of a neural network is one of the most important problems in making neural networks practically useful, but accounts of applications usually sweep these details under the carpet.
Machine LearningConstructing X-of-N Attributes for Decision Tree Learning
36 Citations2000Zijian Zheng
Experimental results reveal that constructing X-of-N attributes can significantly improve the performance of decision tree learning in both artificial and natural domains in terms of higher prediction accuracy and lower theory complexity.