Automatic Wrapper Generation for Web Search Engines

Lecture notes in computer sciencePublished 1 January 2000

Boris Chidlovskii, Jon Ragetli, Maarten de Rijke

DOI10.1007/3-540-45151-x_38

Source URL

Citations10

SJR quartileQ2

SJR score0.35

SNIP0.55

TL;DR

An algorithm to create wrappers automatically, based on an adaptation of the string edit distance is presented, which performs well and can be used for several types of result pages and it requires a minimal amount of interaction with the user.

Abstract

To facilitate effective search on the World Wide Web, several 'meta search engines' have been developed which do not search the Web themselves, but use available search engines to find the required information. By means of wrappers, meta search engines retrieve relevant information from the HTML pages returned by search engines. In this paper we present an algorithm to create such wrappers automatically, based on an adaptation of the string edit distance. Our algorithm performs well; it is quick, it can be used for several types of result pages and it requires a minimal amount of interaction with the user.

Keywords

Computer Science

Wrapper induction for information extraction

1,044 Citations1997Nicholas Kushmerick, Daniel S. Weld

This work introduces wrapper induction, a method for automatically constructing wrappers, and identifies hlrt, a wrapper class that is e(cid:14)ciently learnable, yet expressive enough to handle 48% of a recently surveyed sample of Internet resources.

ACM SIGMOD RecordDatabase techniques for the World-Wide Web

560 Citations1998Daniela Florescu, Alon Y. Levy +1 more

The primary goal of this survey is to classify the different tasks to which database concepts have been applied, and to emphasize the technical innovations that were required to do so.

View PDF

Information SystemsGenerating finite-state transducers for semi-structured data extraction from the Web

414 Citations1998Chun‐Nan Hsu, Ming-Tzung Dung

This paper presents SoftMealy, a novel wrapper representation formalism based on a finite-state transducer and contextual rules that can wrap a wide range of semistructured Web pages because FSTs can encode each different attribute permutation as a path.

Extracting Semistructured Information from the Web.

304 Citations1997J. Hammer, Héctor García-Molina +3 more

A configurable tool for extracting semistructured data from a set of HTML pages and for converting the extracted information into database objects and various ways of improving the functionality of the current prototype are described.

ACM SIGMOD RecordWrapper generation for semi-structured Internet sources

280 Citations1997Naveen Ashish, Craig A. Knoblock

The key idea is to exploit the formatting information in pages from the source to hypothesize the underlying structure of a page and generate a wrapper that facilitates querying of a source and possibly integrating it with other sources.

Elsevier eBooksAlgorithms for Finding Patterns in Strings

249 Citations1990Alfred V. Aho

This chapter discusses the algorithms for solving string-matching problems that have proven useful for text-editing and text-processing applications and several innovative, theoretically interesting algorithms have been devised that run significantly faster than the obvious brute-force method.

Learning to extract text-based information from the World Wide Web

193 Citations1997Stephen Soderland

Webfoot, a preprocessor that parses web pages into logically coherent segments based on page layout cues, is introduced and passed on to CRYSTAL, an NLP system that learns text extraction rules from example.

ProFusion: Intelligent Fusion from Multiple, Distributed Search Engines

128 Citations1996Susan Gauch, Guijun Wang +1 more

View PDF

Towards sophisticated wrapping of web-based information repositories

32 Citations1997Boris Chidlovskii, Uwe M. Borghoff +1 more

IEEE Data(base) Engineering BulletinMediating and Metasearching on the Internet.

31 Citations1998Luis Gravano, Yannis Papakonstantinou

It is suggested that giardial infection may be an important factor contributing to the low nutritional status of many primary schoolchildren in Zimbabwe.