A New Centroid-based Approach for Genre Categorization of Web Pages
LDV-Forum/Journal for language technology and computational linguisticsPublished 1 July 2009Open access
Chaker Jebari
Citations6
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new centroid-based approach for genre catego rization of web pages that is flexible, incremental, refined and combined categorization, which is more suitable for automatic web genre identification.
Abstract
A New Centroid-based Approach for Genre Categorization of Web PagesIn this paper we propose a new centroid-based approach for genre catego rization of web pages.Our approach constructs genre centroids using a set of genre-labeled web pages, called training web pages.The obtained cen troids will be used to classify new web pages.The aim of our approach is to provide a flexible, incremental, refined and combined categorization, which is more suitable for automatic web genre identification.Our approach is flexible because it assigns a web page to all predefined genres with a confi dence score; it is incremental because it classifies web pages one by one; it is refined because each web page either refines the centroids or is discarded as noisy page; finally, our approach combines three different feature sets, i.e.URL addresses, logical structure and hypertext structure.The experiments conducted on two known corpora show that our approach is very fast and outperforms other approaches.
Keywords
Computer Science
Information Processing & ManagementTerm-weighting approaches in automatic text retrieval
9,532 Citations1988Gerard Salton, Chris Buckley
This paper summarizes the insights gained in automatic term weighting, and provides baseline single term indexing models with which other more elaborate content analysis procedures can be compared.
Program electronic library and information systemsAn algorithm for suffix stripping
8,136 Citations1980Martin Porter
An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL, and performs slightly better than a much more elaborate system with which it has been compared.
Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features
7,925 Citations1998Thorsten Joachims
SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.
A Comparative Study on Feature Selection in Text Categorization
4,766 Citations1997Yiming Yang, Jan Pedersen
DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.
Neural ComputationApproximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
3,654 Citations1998Thomas G. Dietterich
This article reviews five approximate statistical tests for determining whether one learning algorithm outperforms another on a particular learning task and measures the power (ability to detect algorithm differences when they do exist) of these tests.
IBM Journal of Research and DevelopmentThe Automatic Creation of Literature Abstracts
3,211 Citations1958H. P. Luhn
In the exploratory research described, the complete text of an article in machine-readable form is scanned by an IBM 704 data-processing machine and analyzed in accordance with a standard program.
Fuzzy Sets and SystemsThe role of fuzzy logic in the management of uncertainty in expert systems
1,282 Citations1983Lotfi A. Zadeh
F fuzzy logic is suggested, which is the logic underlying approximate or, equivalently, fuzzy reasoning, which leads to various basic syllogisms which may be used as rules of combination of evidence in expert systems.
A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization
1,264 Citations1997Thorsten Joachims
A Probabilistic analysis of the Rocchio relevance feedback algorithm, one of the most popular learning methods from information retrieval, is presented in a text categorization framework and suggests that the probabilistic algorithms are preferable to the heuristic Rocchio classifier.
Elsevier eBooksOn Ordered Weighted Averaging Aggregation Operators in Multicriteria Decisionmaking
1,060 Citations1993Ronald R. Yager
Learning to extract symbolic knowledge from the World Wide Web
675 Citations1998Mark Craven, Dan DiPasquo +5 more
The goal of the research described here is to automatically create a computer understandable world wide knowledge base whose content mirrors that of the World Wide Web, and several machine learning algorithms for this task are described.
Automatic detection of text genre
358 Citations1997Brett Kessler, Geoffrey Numberg +1 more
A theory of genres as bundles of facets, which correlate with various surface cues, are proposed, and it is argued that genre detection based on surface cues is as successful as Detection based on deeper structural properties.
Genetic Algorithms and Soft Computing
277 Citations1996Francisco Herrera, José Luís Verdegay
This paper presents fuzzy Logic Based Techniques in Genetic Algorithms in Fuzzy Contexts, a new way of thinking about genetic algorithms that addresses the challenge of fuzzy reality.
arXiv (Cornell University)Automatic Detection of Text Genre
206 Citations1997Brett Kessler, Geoffrey Nunberg +1 more
Journal of the American Society for Information Science and TechnologyLearning to classify documents according to genre
204 Citations2006Aidan Finn, Nicholas Kushmerick
The idea of domain transfer—genre classifiers should be reusable across multiple topics—which does not arise in standard text classification is introduced and different features for building genre classifiers and their ability to transfer across multiple-topic domains are investigated.
Text genre detection using common word frequencies
166 Citations2000Efstathios Stamatatos, Nikos Fakotakis +1 more
It is shown that the most frequent words of the British National Corpus, representing the most Frequence of the written English language, are more reliable discriminators of text genre in comparison to the most frequently spoken words in a training corpus.
Text, speech and language technologyStylistic Experiments in Information Retrieval
151 Citations1999Jussi Karlgren
A discussion on various experiments to utilize stylistic variation among texts for information retrieval purposes and how this affects decision-making in the field of retrieval.
Information Processing & ManagementMultiple sets of features for automatic genre classification of web documents
109 Citations2004Chul Su Lim, Kong‐Joo Lee +1 more
This paper introduces new sets of features specific to web documents, which are extracted from URL and HTML tags within the pages, and concludes which is an appropriate set of features in automatic genre classification of web documents.
Information SciencesEffect of term distributions on centroid-based text categorization
79 Citations2003Verayuth Lertnattee
An approach to apply term distributions, in addition to tf and idf, to improve performance of centroid-based text categorization, and how these term distributions contribute to weight each term in documents is investigated.
Integrating automatic genre analysis into digital libraries
71 Citations2001Andreas Rauber, Alexander Müller-Kögler
An approach to automatically analyze the structure of documents and to integrate this information into an automatically created content-based organization is presented and documents on similar topics, yet representing different genres, are depicted as books in differing colors.
Automatic Identification of Home Pages on the Web
68 Citations2005Alistair Kennedy, Michael Shepherd
Results indicate that the classifier was trained to distinguish home pages from non-home pages and within the home page genre it is able to distinguish personal from corporate home pages, however, organization home pages were more difficult to distinguish from personal and corporateHome pages.
Automatic Identification of Genre in Web Pages
55 Citations2011Marina Santini
It is argued that automatic identification of genre in web pages needs more flexible genre classification schemes, and experiments are described that support this claim.
Effects of web document evolution on genre classification
53 Citations2005Elizabeth Boese, Adele E. Howe
It is found that pages in some genres change rarely if at all and can be used in present-day research experiments without requiring an updated version, and features found to be useful in one corpus do not transfer well to other corpora with different genres.
Genre Classification of Web Pages: User Study and Feasibility Analysis
45 Citations2004Sven Meyer zu Eissen, Benno Stein
Journal of Web EngineeringCybergenre: automatic identification of home pages on the web
40 Citations2004Michael Shepherd, Carolyn Watters +1 more
Conference Committees
39 Citations2007
This paper proposes a low-level representation for Web page genre identification based on character n-grams and produces feature sets of variable-length character n- grams and combines this representation with information about the most frequent HTML-tags.
Webpage Genre Identification Using Variable-Length Character n-Grams
28 Citations2007Ioannis Kanaris, Efstathios Stamatatos
This paper produces feature sets of variable-length character n- grams and combines this representation with information about the most frequent HTML-tags and presents Web page genre identification experiments to improve the best reported results.
The Form is the Substance: Classification of Genres in Text
23 Citations2001Nigel Dewdney, Carol VanEss-Dykema +1 more
Structural classifiers of text types: Towards a novel model of text representation
18 Citations2007Alexander Mehler, Peter Geibel +1 more
LDV-Forum/Journal for language technology and computational linguisticsStructural Classifiers of Text Types: Towards a Novel Model of Text Representation
17 Citations2007Alexander Mehler, Peter Geibel +1 more
For a subset of categories, it is shown that classification can be performed very well by structural dierentia only, and patterns of text structure as determinants of class membership are explored.
Text, speech and language technologyWeb Genre Analysis: Use Cases, Retrieval Models, and Implementation Issues
14 Citations2010Benno Stein, Sven Meyer zu Eissen +1 more
It can be of much help if the retrieval process is capable to address a user’s information need regarding to – what is called here – “genre” or “Web genre”.
Künstliche Intell.On Information Need and Categorizing Search
13 Citations2007Sven Meyer zu Eissen