Overview of the TREC 2006.
Published 1 January 2006
Ellen M. Voorhees
Citations469
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The fourteenth Text REtrieval Conference, TREC 2005, was held at the National Institute of Standards and Technology (NIST) 15 to 18 November 2005 and had 117 participating groups from 23 different countries.
Abstract
The twelfth Text REtrieval Conference, TREC 2003, was held at the National Institute of Standards and Technology (NIST) November 18–21, 2003. The conference was co-sponsored by NIST, the US Department of Defense Advanced Research and Development Activity (ARDA), and the Defense Advanced Research Projects Agency (DARPA).
Keywords
Computer Science
Retrieval evaluation with incomplete information
733 Citations2004Chris Buckley, Ellen M. Voorhees
It is shown that current evaluation measures are not robust to substantially incomplete relevance judgments, and a new measure is introduced that is both highly correlated with existing measures when complete judgments are available and more robust to incomplete judgment sets.
Predicting query performance
669 Citations2002Steve Cronen-Townsend, Yun Zhou +1 more
It is suggested that clarity scores measure the ambiguity of a query with respect to a collection of documents and show that they correlate positively with average precision in a variety of TREC test sets.
How reliable are the results of large-scale information retrieval experiments?
577 Citations1998Justin Zobel
A detailed empirical investigation of the TREC results shows that the measured relative performance of systems appears to be reliable, but that recall is overestimated: it is likely that many relevant documents have not been found.
Information Processing & ManagementVariations in relevance judgments and the measurement of retrieval effectiveness
505 Citations2000Ellen M. Voorhees
Overview of the TREC 2005 Question Answering Track
326 Citations2005Ellen M. Voorhees
In its third year, the TREC question answering track continued to focus on retrieving small snippets of text that contain an answer to a question, however, several new conditions were added to increase the realism, and the difficulty, of the task.
Relevance and Information Behavior.
315 Citations1994Linda Schamber
The author views relevance as a manifestation of human information behavior and excludes work that views relevance only as a matching or computational functions of IR systems.
Overview of the Eighth Text REtrieval Conference (TREC-8).
305 Citations1999Ellen M. Voorhees, Donna Harman
Information Retrieval Experiment
304 Citations1971Karen Spärck Jones
The volume's incohesiveness makes this reviewer question the audience for the volume, and there are a number of quality papers in this volume; perhaps it can be best used to reference the included papers on an individual basis.
The effect of topic set size on retrieval experiment error
246 Citations2002Ellen M. Voorhees, Chris Buckley
Using TREC results to empirically derive error rates based on the number of topics used in a test and the observed difference in the average scores indicates researchers need to take care when concluding one method is better than another, especially if few topics are used.
Text REtrieval ConferenceOverview of the TREC 2004 Web Track.
154 Citations2004Nick Craswell, David Hawking
Studies conducted by the two participating groups compared a search engine using automatic topic distillation features with the same engine with those features disabled in order to determine whether the automatic topic Distillation features assisted the users in the performance of their tasks and whether humans could achieve better results than the automatic system.
CERES (Cranfield University)Aslib Cranfield research project - Factors determining the performance of indexing systems; Volume 1, Design; Part 2, Appendices
153 Citations1966Cyril W. Cleverdon, J. Mills +1 more
An essential requirement of the project involved cooperation of a large number of research scientists, and the response to the request was most satisfactory, and I acknowledge with thanks the generous assistance of some two hundred scientists.
Overview of the TREC 2003 Novelty Track
140 Citations2003Ian Soboroff, Donna Harman
Overview of the TREC 2002 Novelty Track.
133 Citations2002Donna Harman
The NRRC reliable information access (RIA) workshop
113 Citations2004Donna Harman, Chris Buckley
The goal of this workshop (RIA) was to understand how to choose good approaches on a per topic basis for retrieval variability in IR systems.
Why current IR engines fail
106 Citations2004Chris Buckley
University of Glasgow at TREC 2004: Experiments in Web, Robust, and Terabyte Tracks with Terrier.
84 Citations2004Vassilis Plachouras, Ben He +1 more
This work employs a decision mechanism for selecting appropriate retrieval approaches on a per-query basis and uses two pre-retrieval performance predictors and a weighting function recommender mechanism for the mixed query task of the Web track.
TREC 2005 Robust Track Experiments Using PIRCS.
56 Citations2005Kui-Lam Kwok, Laszlo Grunfeld +2 more
Juru at TREC 2004: Experiments with Prediction of Query Difficulty.
19 Citations2004Elad Yom‐Tov, Shai Fine +3 more
The results show that query prediction is indeed ecient in improving retrieval, although further work is needed in order to improve the performance of the prediction algorithms and their uses.
Fondazione Ugo Bordoni at TREC 2004.
17 Citations2004Giambattista Amati, Claudio Carpineto +1 more
The authors' participation in TREC 2004 aims to extend and improve the use of the DFR (Divergence From Randomness) models with Query Expansion (QE) for the robust track, and uses the information theory based function, InfoDFR to predict the AP of queries.
The NTCIR Workshop : the First Evaluation Workshop on Japanese Text Retrieval and Cross-Lingual Information Retrieval
15 Citations1999Noriko Kando, Kazuko Kuriyama +5 more
The outline of the first NTCIR Workshop is introduced, which is the first evaluation workshop designed to enhance research in Japanese text retrieval and cross-lingual information retrieval and some thoughts on the future directions are suggested.
Text REtrieval ConferenceWIDIT in TREC-2004 Genomics, HARD, Robust, and Web tracks
12 Citations2004Kiduk Yang, Ning Yu +5 more
The TREC research group of WIDIT, who engages in examination of information retrieval strategies that can accommodate a variety of data environments and search tasks, participated in the Genomics, HARD, Robust, and Web tracks in TREC-2004.
Text REtrieval ConferenceUIC at TREC-2003: Robust Track.
9 Citations2003Shuang Liu, Chaojing Sun
In TREC 2004, the Database and Information System Lab (DBIS) at University of Illinois at Chicago (UIC) participates in the robust track, which is a traditional ad hoc retrieval task, which the emphasis is based on average effectiveness as well as individual topic effectiveness.
JHU/APL at TREC 2004: Robust and Terabyte Tracks.
7 Citations2004Christine Piatko, James Mayfield +2 more
Although the model does not explicitly incorporate inverse document frequency, it does favor documents that contain more of the rare query terms and can be computed as a similarity measure.
NLPR at TREC 2004: Robust Experiments
4 Citations2004Jin Xu, Jun Zhao +1 more
The Chinese Information Processing group of NLPR participates in TREC and focuses on the Robust Retrieval Track at this year’s conference, based on the previous IR architecture.