Natural language processing for information retrieval
Communications of the ACMPublished 1 January 1996Open access
David Lewis, Karen Spärck Jones
Citations281
SJR quartileQ1
SJR score1.15
SNIP3.34
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The paper considers the new opportunities and challenges presented by the user’s ability to search full text directly (rather than e.g. titles and abstracts), and suggests appropriate approaches to doing this, with a focus on the potential role of natural language processing.
Abstract
article Free Access Share on Natural language processing for information retrieval Authors: David D. Lewis AT&T Bell Labs., Murray Hill, NJ AT&T Bell Labs., Murray Hill, NJView Profile , Karen Spärck Jones Univ. of Cambridge, Cambridge, England Univ. of Cambridge, Cambridge, EnglandView Profile Authors Info & Claims Communications of the ACMVolume 39Issue 1Jan. 1996 pp 92–101https://doi.org/10.1145/234173.234210Published:01 January 1996Publication History 114citation4,937DownloadsMetricsTotal Citations114Total Downloads4,937Last 12 Months555Last 6 weeks99 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF
Keywords
Computer Science
Journal of the American Society for Information ScienceImproving retrieval performance by relevance feedback
1,504 Citations1990Gerard Salton, Chris Buckley
Relevance feedback is an automatic process, introduced over 20 years ago, designed to produce query formulations following an initial retrieval operation to demonstrate the effectiveness of the various methods.
Communications of the ACMThe vocabulary problem in human-system communication
1,497 Citations1987George W. Furnas, Thomas K. Landauer +2 more
It is shown how this fundamental property of language limits the success of various design methodologies for vocabulary-driven interaction, and an optimal strategy, unlimited aliasing, is derived and shown to be capable of several-fold improvements.
A stochastic parts program and noun phrase parser for unrestricted text
974 Citations1988Kenneth Church
A program that tags each word in an input sentence with the most likely part of speech has been written and performance is encouraging; a 400-word sample is presented and is judged to be 99.5% correct.
Journal of Computational and Applied MathematicsDepartment of Computer Science
618 Citations1991Chakchai So-in, Raj Jain +2 more
This seminar overviews this recent research trend and studies the characteristics of successful research in this space, to understand whether this trend will have a long-term impact on the design and implementation of large-scale computing systems.
ACM Transactions on Information SystemsEvaluation of an inference network-based retrieval model
577 Citations1991Howard R. Turtle, W. Bruce Croft
Network representations show promise as mechanisms for inferring probable relationships between documents and queries and have been used in information retrieval since at least the early 1960s.
An evaluation of phrasal and clustered representations on a text categorization task
547 Citations1992David Lewis
It is shown that optimal effectiveness occurs when using only a small proportion of the indexing terms available, and that effectiveness peaks at a higher feature set size and lower effectiveness level for a syntactic phrase indexing than for word-based indexing.
Noun classification from predicate-argument structures
543 Citations1990Donald Hindle
The resulting quasi-semantic classification of nouns demonstrates the plausibility of the distributional hypothesis, and has potential application to a variety of tasks, including automatic indexing, resolving nominal compounds, and determining the scope of modification.
ACM Transactions on Information SystemsLexical ambiguity and information retrieval
407 Citations1992Robert Krovetz, W. Bruce Croft
An analysis of lexical ambiguity in information retrieval test collections and experiments to determine the utility of word meanings for separating relevant from nonrelevant documents show that there is considerable ambiguity even in a specialized database.
Information Processing & ManagementOverview of the Second Text Retrieval Conference (TREC-2)
310 Citations1995Donna Harman
This conference, co-sponsored by ARPA and NIST, brought together information retrieval researchers to discuss their system results on the new TIPSTER test collection, and represented a breakthrough in cross-system evaluation in information retrieval.
Information Retrieval Experiment
304 Citations1971Karen Spärck Jones
The volume's incohesiveness makes this reviewer question the audience for the volume, and there are a number of quality papers in this volume; perhaps it can be best used to reference the included papers on an individual basis.
Communications of the ACMAnother look at automatic text-retrieval systems
275 Citations1986Gerard Salton
Evidence from available studies comparing manual and automatic text-retrieval systems does not support the conclusion that intellectual content analysis produces better results than comparable automatic systems.
The use of phrases and structured queries in information retrieval
265 Citations1991W. Bruce Croft, Howard R. Turtle +1 more
The results show that using phrases in this way can improve performance, and that phrases that are automatically extracted from a natural language query perform nearly as well as manually selected phrases.
Communications of the ACMSCISOR: extracting information from on-line news
217 Citations1990Paul S. Jacobs, Lisa F. Rau
The future of natural language text processing is examined in the SCISOR prototype, drawing on artificial intelligence techniques, and applying them to financial news items through a combination of bottom-up and top-down processing.
Cornell University eBooksExperiments in Automatic Phrase Indexing For Document Retrieval: A Comparison of Syntactic and Non-Syntactic Methods
185 Citations1987Joel L. Fagan
A 78-year-old man with palpitations, dizziness, and presyncope was referred to the authors' institution for evaluation and a diagnosis of right ventricular dysplasia was made, and the patient underwent an electrophysiological study to evaluate hisPalpitations and Presyncope.
Information Processing & ManagementModels for retrieval with probabilistic indexing
177 Citations1989Norbert Fuhr
Three retrieval models for probabilistic indexing are described along with evaluation results for each, including the binary independence indexing (BII) model, which is a generalized version of the Maron and Kuhns indexing model.
The Knowledge Engineering ReviewNatural language interfaces to databases
145 Citations1990Ann Copestake, Karen Spärck Jones
The paper discusses the issues, both general to language and task-specific, involved in front end design and the way these have been addressed, concentrating on the work of the last decade, indicating that future progress will depend on the one hand on general advances in natural language processing, and on the other on expanding the capabilities of traditional databases.
Computational LinguisticsEvaluating message understanding systems: an analysis of the third message understanding conference (MUC-3)
139 Citations1993Nancy Chinchor, David Lewis +1 more
The purpose, history, and methodology of the conference are reviewed, the participating systems are summarized, issues of measuring system effectiveness are discussed, the linguistic phenomena tests are described, and a critical look at the evaluation in terms of the lessons learned is provided.
Journal of DocumentationAUTOMATIC SEARCH TERM VARIANT GENERATION
127 Citations1984Karen Spärck Jones, John Tait
The paper describes research designed to improve automatic pre‐coordinate term indexing by applying powerful general‐purpose language analysis techniques to identify term sources in requests, and to generate variant expressions of the concepts involved for document text searching.
A Comparison of Internet Resource Discovery Approaches
98 Citations1992Michael F. Schwartz, Alan Emtage +2 more
This paper presents a taxonomy of approaches to resource discovery, and uses this taxonomy to compare a number of resource discovery systems, and examine several gateways between existing systems.
ScienceGlobal Text Matching for Information Retrieval
82 Citations1991Gerard Salton, Chris Buckley
An approach is outlined for the retrieval of natural language texts in response to available search requests and for the recognition of content similarities between text excerpts that appears to outperform other currently available methods.
Information Processing & ManagementGrammatically-based automatic word class formation
79 Citations1975Lynette Hirschman, Ralph Grishman +1 more
An automatic procedure which uses the syntactic relations as the basis for grouping words into classes that can be used to describe the informational patterns that occur in texts of the subfield, to disambiguate parses of a sentence, and perhaps to improve the performance of current information retrieval systems.
Experiments with query acquisition and use in document retrieval systems
69 Citations1989W. Bruce Croft, R. Das
A study evaluating how easily enhanced queries can be acquired from users and how effectively this additional knowledge can be used in retrieval indicates that significant effectiveness benefits can be obtained through the acquisition of domain concepts related to query concepts.
Information Processing & ManagementTopic parsing: Accounting for text macro structures in full-text analysis
57 Citations1990Udo Hahn
This paper introduces a parser which is based on the conceptual knowledge of its domain and is organized as a collection of distributed lexicalized grammar modules (word experts) which communicate through message-passing.
The importance of proper weighting methods
55 Citations1993Chris Buckley
Evidence is presented that good weighting methods are more important than the feature selection process and it is suggested that the two need to go hand-in-hand in order to be effective.
Text REtrieval ConferenceDesign and evaluation of the CLARIT-TREC-2 system
53 Citations1993David A. Evans, Robert G. Lefferts
The CLarIT team used the opportunity of the TREC-2 evaluations to explore several facets of the CLARIT system, and focused its attention on evaluating: 1-fully-automatic processing of topics and potentially-relevant documents and; 2-Topic/query augmentation using Clarit thesaurus techniques.
From N-grams to collocations
48 Citations1991Frank Smadja
These methods are implemented in an added third stage to Xtract that examines the set of collocations retrieved during the previous two stages to both filter out a number of invalid collocations and add useful syntactic information to the retained ones.
An evaluation of query processing strategies using the TIPSTER collection
27 Citations1993James P. Callan, W. Bruce Croft
This paper describes several methods of constructing queries for the INQUERY information retrieval system, and then evaluates those methods on the TIPSTER document collection.
Recent developments in natural language text retrieval
21 Citations1993Tomek Strzalkowski, Jose Perez Carballo
This paper reports on some revent developments in the natural language text retrieval system, a traditional statistical engine which builds inverted index files from pre-processed documents and then searches and ranks the documents in response to user queries.
Journal of Information ScienceSearch term relevance weighting- some recent results
14 Citations1979Karen Spärck Jones
The results show that relevance weighting can improve retrieval performance substantially in the stringent conditions implied by large document sets and little relevance assessment information.
Information Processing & ManagementFashionable trends and feasible strategies in information management
9 Citations1988Karen Spärck Jones
Current trends in information management are analyzed, the problems they involve, and some strategies for tackling these problems are suggested, including linguistically oriented knowledge and inference.