A Quantitative Study of Small Disjuncts

Published 30 July 2000

Gary M. Weiss, Haym Hirsh

Citations67

TL;DR

This paper presents a quantitative measure for evaluating the effect of small disjuncts on learning and uses it to analyze 30 benchmark datasets and comes up with several interesting results.

Abstract

Systems that learn from examples often express the learned concept in the form of a disjunctive description. Disjuncts that correctly classify few training examples are known as small disjuncts and are interesting to machine learning researchers because they have a much higher error rate than large disjuncts. Previous research has investigated this phenomenon by performing ad hoc analyses of a small number of datasets. In this paper we present a quantitative measure for evaluating the effect of small disjuncts on learning and use it to analyze 30 benchmark datasets. We investigate the relationship between small disjuncts and pruning, training set size and noise, and come up with several interesting results. Introduction Systems that learn from examples often express the learned concept as a disjunction. The size of a disjunct is defined as the number of training examples that it correctly classifies (Holte, Acker, and Porter 1989). A number of empirical studies have demons...

Keywords

Computer Science

C4.5: Programs for Machine Learning

23,665 Citations1992J. R. Quinlan

A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.

Elsevier eBooksFast Effective Rule Induction

3,767 Citations1995William W. Cohen

This paper evaluates the recently-proposed rule learning algorithm IREP on a large and diverse collection of benchmark problems, and proposes a number of modifications resulting in an algorithm RIPPERk that is very competitive with C4.5 and C 4.5rules with respect to error rates, but much more efficient on large samples.

Concept learning and the problem of small disjuncts

342 Citations1989Robert C. Holte, Liane Acker +1 more

Various approaches to this problem are evaluated, including the novel approach of using a bias different than the "maximum generality" bias, which prove partly successful, but the problem of small disjuncts remains open.

A simple, fast, and effective rule learner

330 Citations1999William W. Cohen, Yoram Singer

Elsevier eBooksLearning with Rare Cases and Small Disjuncts

86 Citations1995Gary M. Weiss

One key conclusion is that when low levels of attribute noise are applied only to the training set (the ability to learn the correct concept is being evaluated), rare cases within a domain are primarily responsible for making learning difficult.

Elsevier eBooksSmall Disjuncts in Action: Learning to Diagnose Errors in the Local Loop of the Telephone Network

71 Citations1993Andrea Danyluk, Foster Provost

The need for smaller disjuncts is a major reason that it is difficult to learn from errorful data in this domain of NYNEX MAX, an expert system that diagnoses the local loop in a telephone network.

When small disjuncts abound, try lazy learning: A case study

40 Citations1997Antal van den Bosch, Ton Weijters +2 more

The measures introduced in this paper are useful for predicting the suitedness of lazy learning in general and are illustrated by a sample language task, viz. word pronunciation.

View PDF

The problem with noise and small disjuncts

34 Citations1994Gary M. Weiss

It is shown that when noise is added to two real-world domains, a signiﬁcant, and disproportionate number of the total errors are contributed by the small disjuncts; thus, in the presence of noise, it is the small disjuncts that are primarily responsible for the poor predictive accuracy of the learned concept.

View PDF

The MIT Press eBooksReducing the Small Disjuncts Problem by Learning Probabilistic Concept Descriptions

19 Citations1995Ali, Kamal M., Pazzani, Michael J.

The system HYDRA is introduced which learns probabilistic rela tional concepts and reduces contribution of error from small disjuncts and is demonstrated the reduction of the small disjuncts problem on various relational and attribute-value do mains.

View PDF

ViewA quantitive study of small disjuncts: experiments and results

4 Citations2000Gary M. Weiss, Haym Hirsh

This paper presents a quantitative measure for evaluating the effect of small disjuncts on learning and uses it to analyze 30 benchmark datasets and comes up with several interesting results.

View PDF