A novel method for the evaluation of Boolean query effectiveness across a wide operational range
Published 1 July 2000
Eero Sormunen
Citations11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new laboratory-based evaluation method for Boolean IR systems is proposed, based on a controlled formulation of inclusive query plans, on an automatic conversion of query plans into elementary queries, and on combining elementary queries into optimal queries at standard points of operation.
Abstract
Traditional methods for the system-oriented evaluation of Boolean IR system suffer from validity and reliability problems. Laboratory-based research neglects the searcher and studies suboptimal queries. Research on operational systems fails to make a distinction between searcher performance and system performance. This approach is neither capable of measuring performance at standard points of operation (e.g. across R0.0-R1.0).
Keywords
Computer Science
Journal of the Operational Research SocietyKnapsack Problems: Algorithms and Computer Implementations
3,163 Citations1991David Connolly, Silvano Martello +1 more
This paper focuses on the part of the knapsack problem where the problem of bin packing is concerned and investigates the role of computer codes in the solution of this problem.
Introduction to Modern Information Retrieval
921 Citations1999Gobinda Chowdhury
It is this book's contention that it also benefits information professionals to learn the theory, techniques and tools that constitute the traditional approaches to the organization and processing of information.
Communications of the ACMAn evaluation of retrieval effectiveness for a full-text document-retrieval system
666 Citations1985David C. Blair, M. E. Maron
An evaluation of a large, operational full-text document-retrieval system shows the system to be retrieving less than 20 percent of the documents relevant to a particular search.
Aslib ProceedingsThe CRANFIELD TESTS ON INDEX LANGUAGE DEVICES
470 Citations1967Cyril W. Cleverdon
It appeared that the most important consideration was the specificity of the index terms; within the context of the conditions existing in this test, single‐word terms were more effective than concept terms or a controlled vocabulary.
Information Retrieval Experiment
304 Citations1971Karen Spärck Jones
The volume's incohesiveness makes this reviewer question the audience for the volume, and there are a number of quality papers in this volume; perhaps it can be best used to reference the included papers on an individual basis.
Communications of the ACMAnother look at automatic text-retrieval systems
275 Citations1986Gerard Salton
Evidence from available studies comparing manual and automatic text-retrieval systems does not support the conclusion that intellectual content analysis produces better results than comparable automatic systems.
Information retrieval systems: characteristics, testing and evaluation
248 Citations1979F. W. Lancaster
Information retrieval systems: characteristics, testing, and evaluation, Information retrieval systems : characteristics, test- and evaluation.
Journal of the American Society for Information ScienceA study of information seeking and retrieving. I. Background and methodology
242 Citations1988Tefko Saračević, Paul B. Kantor +2 more
Information Processing & ManagementThe pragmatics of information retrieval experimentation, revisited
228 Citations1992Jean Tague‐Sutcliffe
Evaluation of evaluation in information retrieval
227 Citations1995Tefko Saračević
A critical and historical analysis of evaluations of IR systems and processes, and issues related to systems under evaluation, and evaluation criteria, measures, measuring instruments, and methodologies are examined.
Medical Entomology and ZoologyOnline Information Retrieval: Concepts, Principles and Techniques
148 Citations1986Stephen P. Harter
The best criterion to decide which patient with asymmetric hearing loss should be referred to MRI is based on the average bone conduction threshold difference at multiple (two or three) higher frequencies.
Journal of the American Society for Information ScienceSearchers' selection of search keys: I. The selection routine
108 Citations1991Raya Fidel
The selection routine is a formal decision tree that represents the intuitive rules searchers use when they select search keys, textwords or descriptors and is the first formal presentation of human knowledge that can be incorporated into the knowledge base of intermediary expert systems.
Journal of the American Society for Information ScienceA new comparison between conventional indexing (MEDLARS) and automatic text processing (SMART)
90 Citations1972Gerard Salton
It is concluded that no technical justification exists for maintaining controlled, manual indexing in operational retrieval environments through conventional retrieval and automatic text analysis methods (SMART).
Medical Entomology and ZoologyInformation Retrieval Today
85 Citations1993F. W. Lancaster, Amy J. Warner
A Method for Measuring Wide Range Performance of Boolean Queries in Full-Text Databases
55 Citations2000Eero Sormunen
A selection of photographs from the archives of the Natural History Museum of London, showing the evolution of the institution over 175 years and the collection's decline in importance and popularity.
Information Processing & ManagementConsistency in the selection of search concepts and search terms
45 Citations1995Mirja Iivonen
Journal of the American Society for Information ScienceBoolean search: Current state and perspectives
42 Citations1999Valery I. Frants, Jacob Shapiro +2 more
The article analyzes existing criticisms of operational systems and points out some of their positive features, and provides several conclusions about using Boolean logic in developing multiversion IR systems.
Journal of the American Society for Information ScienceAn evaluation of interactive Boolean and natural language searching with an online medical textbook
40 Citations1995William Hersh, David H. Hickam
A number of problems with traditional measures of retrieval evaluation when applied to the interactive search setting are found.
Journal of the American Society for Information ScienceThe Medline/full-text research project
27 Citations1991Emma Jean McKinin, MaryEllen Sievert +2 more
It was found that full-text identified significantly more relevant articles than did the indexed file, MEDLINE, and the more current issue of a journal was just as likely to appear in MEDLINE as in one of the full- Text files.
A deductive data model for query expansion
26 Citations1996Kalervo J rvelin, Jaana Kristensen +3 more
A deductive data model for concept-based query expansion based on three abstraction levels: the conceptual, the expression and the occurrence level, which supports a concept- based query expantion and formulation tool, the ExpansionTool, for environments providing heterogeneous IR systems.
Journal of the American Society for Information ScienceSearch term combinations and retrieval overlap: A proposed methodology and case study
20 Citations1990Stephen P. Harter
Online information retrieval
20 Citations1979Carol Hansen Fenichel, Charles T. Meadow
Information Processing & ManagementFreestyle vs. Boolean: A comparison of partial and exact match retrieval systems
17 Citations1998Lee Anne H. Paris, Helen R. Tibbo
Although Boolean searching has been the standard model for commercial information retrieval systems for the past three decades, natural language input and partial-match weighted retrieval have recently emerged from the laboratories to become a searching option in several well-known online systems.