Hierarchical classification
Published 1 January 2006
Nicolò Cesa‐Bianchi, Claudio Gentile, Luca Zaniboni
Citations143
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A refined evaluation scheme is introduced which turns the hierarchical SVM classifier into an approximator of the Bayes optimal classifier with respect to a simple stochastic model for the labels.
Abstract
We study hierarchical classification in the general case when an instance could belong to more than one class node in the underlying taxonomy. Experiments done in previous work showed that a simple hierarchy of Support Vectors Machines (SVM) with a top-down evaluation scheme has a surprisingly good performance on this kind of task. In this paper, we introduce a refined evaluation scheme which turns the hierarchical SVM classifier into an approximator of the Bayes optimal classifier with respect to a simple stochastic model for the labels. Experiments on synthetic datasets, generated according to this stochastic model, show that our refined algorithm outperforms the simple hierarchical SVM. On real-world data, however, the advantage brought by our approach is a bit less clear. We conjecture this is due to a higher noise rate for the training labels in the low levels of the taxonomy.
Keywords
Computer Science
Machine LearningSupport-Vector Networks
33,035 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Springer series in statisticsThe Elements of Statistical Learning
24,344 Citations2001Trevor Hastie, J. Friedman +1 more
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
4,863 Citations1999John Platt
The output of a lassi(cid:12)er should be a alibrated posterior probability to enable post-pro essing and a method to train a kernel lassi with a logit link and a regularized maximum likelihood is proposed.
ePrints Soton (University of Southampton)An Introduction to Support Vector Machines
4,120 Citations2000Nello Cristianini, John Shawe‐Taylor
Predicting good probabilities with supervised learning
1,617 Citations2005Alexandru Niculescu-Mizil, Rich Caruana
The relationship between the predictions made by different learning algorithms and true posterior probabilities is examined, showing that maximum margin methods such as boosted trees and boosted stumps push probability mass away from 0 and 1 yielding a characteristic sigmoid shaped distortion in the predicted probabilities.
Hierarchically Classifying Documents Using Very Few Words
840 Citations1997Daphne Koller, Mehran Sahami
This work proposes an approach that utilizes the hierarchical topic structure to decompose the classification task into a set of simpler problems, one at each node in the classification tree, which can be solved accurately by focusing only on a very small set of features, those relevant to the task at hand.
Hierarchical classification of Web content
805 Citations2000Susan Dumais, Hao Chen
This paper explores the use of hierarchical structure for classifying a large, heterogeneous collection of web content using support vector machine (SVM) classifiers, which have been shown to be efficient and effective for classification, but not previously explored in the context of hierarchical classification.
Improving Text Classification by Shrinkage in a Hierarchy of Classes
478 Citations2022Andrew McCallum, Roni Rosenfeld +2 more
This paper shows that the accuracy of a naive Bayes text classi(cid:12)er can be significantly improved by taking advantage of a hierarchy of classes, and adopts an established statistical technique called shrinkage that smoothes parameter estimates of a data-sparse child with its parent in order to obtain more robust parameter estimates.
Information RetrievalHierarchical Text Categorization Using Neural Networks
248 Citations2002Miguel E. Ruiz, Padmini Srinivasan
The results show that the use of the hierarchical structure improves text categorization performance with respect to an equivalent flat model, and the optimized Rocchio algorithm achieves a performance comparable with that of the Hierarchical Mixture of Experts model.
Large margin hierarchical classification
216 Citations2004Ofer Dekel, Joseph Keshet +1 more
This work presents an algorithmic framework for supervised classification learning where the set of labels is organized in a predefined hierarchical structure encoded by a rooted tree which induces a metric over the label set.
Incremental Algorithms for Hierarchical Classification
113 Citations2004Nicolò Cesa‐Bianchi, Claudio Gentile +1 more
A new hierarchical loss function, the H-loss, is introduced, implementing the simple intuition that additional mistakes in the subtree of a mistaken class should not be charged for, based on a probabilistic data model introduced in earlier work.
Learning hierarchical multi-category text classification models
91 Citations2005Juho Rousu, Craig Saunders +2 more
This work presents an efficient optimization algorithm based on incremental conditional gradient ascent in single-example subspaces spanned by the marginal dual variables that can feasibly optimize training sets of thousands of examples and classification hierarchies consisting of hundreds of nodes.
European Conference on Artificial IntelligenceTurning Yahoo to Automatic Web-Page Classifier.
62 Citations1998Dunja Mladenić
This preliminary study suggests that in the second trimester, Angptl6 levels are higher in women with subsequent PIH.
Lecture notes in computer scienceAn Online Algorithm for Hierarchical Phoneme Classification
45 Citations2005Ofer Dekel, Joseph Keshet +1 more
A new online algorithm for solving the hierarchical classification problem and worst-case loss analysis for the algorithm is described and the merits of the approach are demonstrated by applying the algorithm to synthetic data and as well as speech data.
Lecture notes in computer scienceRegret Bounds for Hierarchical Classification with Linear-Threshold Functions
16 Citations2004Nicolò Cesa‐Bianchi, Alex Conconi +1 more
An incremental algorithm using a linear-threshold classifier at each node of the taxonomy is introduced using a hierachical and parametric data model and is proved to prove a bound on the probability that the algorithm guesses the wrong multilabel for a random instance when the true model parameters are known.