How to build a WebFountain: An architecture for very large-scale text analytics
IBM Systems JournalPublished 1 January 2004
Daniel Gruhl, L. Chavet, David Gibson, Jeannie Meyer, Pradipta Kumar Pattanayak, Andrew Tomkins
Citations95
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper surveys the high-level decisions made in creating a platform for very large-scale text analytics applications and creates an extensible set of hosted Web services containing information that drives end-user applications.
Abstract
WebFountain is a platform for very large-scale text analytics applications. The platform allows uniform access to a wide variety of sources, scalable system-managed deployment of a variety of document-level "augmenters" and corpus-level "miners," and finally creation of an extensible set of hosted Web services containing information that drives end-user applications. Analytical components can be authored remotely by partners using a collection of Web service APIs (application programming interfaces). The system is operational and supports live customers. This paper surveys the high-level decisions made in creating such a system.
Keywords
Computer Science
ACM SIGMOD RecordA case for redundant arrays of inexpensive disks (RAID)
2,301 Citations1988David A. Patterson, Garth A. Gibson +1 more
ACM Computing SurveysThe Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty
1,341 Citations1980Lee D. Erman, Frederick Hayes‐Roth +2 more
The characteristics of the speech problem in particular, the special kinds of problem-solving uncertainty in that domain, the structure of the Hearsay-II system developed to cope with that uncertainty, and the relationship between Hearsey-II's structure and those of other speech-understanding systems are discussed.
The anatomy of the grid: enabling scalable virtual organizations
1,095 Citations2005Ian Foster
The term "the Grid" was coined in the mid-1990s to denote a proposed distributed computing infrastructure for advanced science and engineering but the term has also been conflated, at least in popular perception, to embrace everything from advanced networking to artificial intelligence.
International Journal on Digital LibrariesThe Lorel query language for semistructured data
1,067 Citations1997Serge Abiteboul, Dallan Quass +3 more
The main novelties of the Lorel language are the extensive use of coercion to relieve the user from the strict typing of OQL, which is inappropriate for semistructured data; and powerful path expressions, which permit a flexible form of declarative navigational access and are particularly suitable when the details of the structure are not known to the user.
Enhanced hypertext categorization using hyperlinks
775 Citations1998Soumen Chakrabarti, Byron Dom +1 more
This work has developed a text classifier that misclassified only 13% of the documents in the well-known Reuters benchmark; this was comparable to the best results ever obtained and its technique also adapts gracefully to the fraction of neighboring documents having known topics.
Disk Scheduling Revisited
276 Citations1990Margo Seltzer, Peter Chen +1 more
This work has analyzed traditional disk scheduling techniques, which attempt to optimize head movement and guarantee fairness in response time, in the presence of long disk queues and proposes two algorithms which take rotational latency into account.
UTF-8, a transformation format of ISO 10646
220 Citations1998F. Yergeau
This memo updates and replaces RFC 2044, in particular addressing the question of versions of the relevant standards, and has the characteristic of preserving the full US-ASCII range, providing compatibility with file systems, parsers and other software that rely on US- ASCII values but are transparent to other values.
Adaptive Query Processing: Technology in Evolution.
213 Citations2000Joseph M. Hellerstein, Michael J. Franklin +6 more
A survey of prior work on adaptive query processing focuses on three characterizations of adaptivity: the frequency of adaptivity, the effects of adaptivity, and the extent of adaptivity, focusing on three characterizations of adaptivity.
Computer NetworksWebBase: a repository of Web pages
187 Citations2000Jun Hirai, Sriram Raghavan +2 more
This paper focuses on the storage manager module, and illustrates how traditional techniques for storage and indexing can be tailored to meet the requirements of a web repository.
Computer Networks and ISDN SystemsThe Connectivity Server: fast access to linkage information on the Web
187 Citations1998Krishna Bharat, Andrei Broder +3 more
A server that provides linkage information for all pages indexed by the AltaVista search engine and can produce the entire neighbourhood of L up to a given distance, and envisage numerous other applications such as ranking, visualization, and classification.
Computer Networks and ISDN SystemsApplications of a Web query language
103 Citations1997Gustavo O. Arocena, Alberto O. Mendelzon +1 more
This paper reports on the experience using WebSQL, a high level declarative query language for extracting information from the Web that takes advantage of multiple index servers without requiring users to know about them, and integrates full-text with topology-based queries.
Shortest substring ranking (MultiText experiments for TREC-4)
59 Citations1995Charles L. A. Clarke, Gordon V. Cormack +1 more
To adress the TREC-4 topics, this work used a precise query language that yields and combines arbitrary intervals of texts rather than pre-defined units like words and documents to compare favourably with the median average precision.
Vinci
42 Citations2001Rakesh Agrawal, Roberto J. Bayardo +2 more
It is shown how systems architected with Vinci are developed quickly, scaled effortlessly, and easily moved from prototype to production.
Computer NetworksSqueal: a structured query language for the Web
38 Citations2000Ellen Spertus, Lynn Andrea Stein
A programming language, Squeal, is implemented that facilitates structure-based queries on the Web as if it were in a standard relational database and shows the ease of writing structure- based information tools in Squeal.
Lecture notes in computer scienceEfficient queries over web views
26 Citations1998Giansalvatore Mecca, Alberto O. Mendelzon +1 more
Massive computingAlgorithmic Aspects of Information Retrieval on the Web
12 Citations2002Andrei Broder, Monika Henzinger
This chapter will survey emerging techniques for information retrieval in the Web context and discuss some of the pertinent open problems.
DSpace@MIT (Massachusetts Institute of Technology)The search for meaning in large text databases
4 Citations2000Daniel Gruhl, Marvin Minsky
This thesis examines a system that employs richer representation of the texts in its corpus that employs a variety of methods ranging from statistical examination to natural-language parsing to query expansion through specific-purpose knowledge bases.