The form is the substance
Published 1 January 2001Open access
Nigel Dewdney, Carol VanEss-Dykema, Richard MacMillan
Citations74
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work investigates text classification by format style, i.e. "genre", and demonstrates, by complementing topic classification, that it can significantly improve retrieval of information.
Abstract
Categorization of text in IR has traditionally focused on topic. As use of the Internet and e-mail increases, categorization has become a key area of research as users demand methods of prioritizing documents. This work investigates text classification by format style, i.e. "genre", and demonstrates, by complementing topic classification, that it can significantly improve retrieval of information. The paper compares use of presentation features to word features, and the combination thereof, using Naïve Bayes, C4.5 and SVM classifiers. Results show use of combined feature sets with SVM yields 92% classification accuracy in sorting seven genres.
Keywords
Computer Science
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
Data Mining and Knowledge DiscoveryA Tutorial on Support Vector Machines for Pattern Recognition
16,433 Citations1998Christopher J. C. Burges
There are several arguments which support the observed high accuracy of SVMs, which are reviewed and numerous examples and proofs of most of the key theorems are given.
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
Program electronic library and information systemsAn algorithm for suffix stripping
8,136 Citations1980Martin Porter
An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL, and performs slightly better than a much more elaborate system with which it has been compared.
Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features
7,925 Citations1998Thorsten Joachims
SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.
Cambridge University Press eBooksVariation across Speech and Writing
5,128 Citations1988Douglas Biber
In Variation Across Speech and Writing, six dimensions of variation are identified through a factor analysis, on the basis of linguistic co-occurence patterns, and the resulting model of variation provides for the description of the distinctive linguistic characteristic of any spoken or written text and enables reconciliation of the contradictory conclusions reached in previous research.
A Comparative Study on Feature Selection in Text Categorization
4,766 Citations1997Yiming Yang, Jan Pedersen
DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.
ePrints Soton (University of Southampton)An Introduction to Support Vector Machines
4,120 Citations2000Nello Cristianini, John Shawe‐Taylor
Computational LinguisticsA maximum entropy approach to natural language processing
3,120 Citations1996Adam Berger, Vincent J. Della Pietra +1 more
A maximum-likelihood approach for automatically constructing maximum entropy models is presented and how to implement this approach efficiently is described, using as examples several problems in natural language processing.
Training support vector machines: an application to face detection
2,496 Citations2002E. Osuna, Robert M. Freund +1 more
A decomposition algorithm that guarantees global optimality, and can be used to train SVM's over very large data sets is presented, and the feasibility of the approach on a face detection problem that involves a data set of 50,000 data points is demonstrated.
A Bayesian Approach to Filtering Junk E-Mail
1,153 Citations1998Mehran Sahami, Susan Dumais +2 more
This work examines methods for the automated construction of filters to eliminate such unwanted messages from a user’s mail stream, and shows the efficacy of such filters in a real world usage scenario, arguing that this technology is mature enough for deployment.
HAL (Le Centre pour la Communication Scientifique Directe)Conversation and community: chat in a virtual world
428 Citations1999Lynn Cherny
Conversation and Community: Discourse in a Social MUD is an examination of the Speech community in an Internet "virtual community" based on ethnographic research on a community of users of a MUD.
Learning Rules that Classify E-Mail
400 Citations1996William W. Cohen
Two methods for learning text classifiers are compared on classification problems that might arise in filtering and filing personM e-mail messages: a "traxiitionM IR" method based on TF-IDF weighting, and a new method for learning sets of "keyword-spotting rules" based on the RIPPER rule learning algorithm.
Automatic detection of text genre
358 Citations1997Brett Kessler, Geoffrey Numberg +1 more
A theory of genres as bundles of facets, which correlate with various surface cues, are proposed, and it is argued that genre detection based on surface cues is as successful as Detection based on deeper structural properties.
Program electronic library and information systemsAn algorithm for suffix stripping
311 Citations2006Matthew Porter
Recognizing text genres with simple metrics using discriminant analysis
293 Citations1994Jussi Karlgren, Douglass R. Cutting
A simple method for categorizing texts into pre-determined text genre categories using the statistical standard technique of discriminant analysis is demonstrated with application to the Brown corpus.
arXiv (Cornell University)Automatic Detection of Text Genre
206 Citations1997Brett Kessler, Geoffrey Nunberg +1 more
Text genre detection using common word frequencies
166 Citations2000Efstathios Stamatatos, Nikos Fakotakis +1 more
It is shown that the most frequent words of the British National Corpus, representing the most Frequence of the written English language, are more reliable discriminators of text genre in comparison to the most frequently spoken words in a training corpus.
Journal of the American Society for Information ScienceAutomatic classification of E-mail messages by message type
12 Citations1997A D May
A system that automatically classifies E-mail messages in the HUMANIST electronic discussion group into one of four classes, Question, Response, Announcement, or Administrative, is described.
Learning Text Filtering Preferences
11 Citations1996Anandeep Pannu, Katia Sycara
A reusable agent that learns a model of the user’s research interests for filtering conference announcements and request for proposals from the Web and its performance at startup is quite high.
On automatic filtering of multilingual texts
7 Citations2002Douglas W. Oard, N. DeClaris +2 more
This study introduces novel approaches for filtering texts regardless of their source language and presents three alternative approaches to selecting texts from a multilingual information stream which represent a logical evolution from existing techniques in related disciplines.
TyPTex: generic features for text profiler
5 Citations2000Gabriel Illouz, Benoît Habert +5 more
This work is implementing profiling tools and developing an associated methodology within the ELRA benchmark named Contribution to the construction of corpora of contemporary French that yields constraints for corpus profiling architectures.