CLEF Methodology and Metrics

Lecture notes in computer sciencePublished 1 January 2002

Martín Braschler, C Peters

DOI10.1007/3-540-45691-0_37

Source URL

Citations19

SJR quartileQ2

SJR score0.35

SNIP0.55

TL;DR

The organization of the CLEF 2001 evaluation campaign is described, the guidelines given to participants are outlined, and the techniques and measures used in CLEF campaigns for result calculation and analysis are explained.

Abstract

We describe the organization of the CLEF 2001 evaluation campaign, outline the guidelines given to participants, and explain the techniques and measures used in CLEF campaigns for result calculation and analysis.

Keywords

Computer Science

Aslib ProceedingsThe CRANFIELD TESTS ON INDEX LANGUAGE DEVICES

470 Citations1967Cyril W. Cleverdon

It appeared that the most important consideration was the specificity of the index terms; within the context of the conditions existing in this test, single‐word terms were more effective than concept terms or a controlled vocabulary.

Lecture notes in computer scienceThe Philosophy of Information Retrieval Evaluation

385 Citations2002Ellen M. Voorhees

The fundamental assumptions and appropriate uses of the Cranfield paradigm, especially as they apply in the context of the evaluation conferences, are reviewed.

Lecture notes in computer scienceCross-Language Information Retrieval and Evaluation

151 Citations2001C Peters, Kraaij, Wessel +2 more

When you read more every page of this cross language information retrieval and evaluation, what you will obtain is something great.

View PDF

Computers & Mathematics with ApplicationsMultimedia information retrieval: Content-based information retrieval from large text and audio databases

76 Citations1998

Text REtrieval ConferenceThe TREC-2001 Cross-Language Information Retrieval Track: Searching Arabic using English, French or Arabic Queries

64 Citations2001Fredric C. Gey, Douglas W. Oard

A variety of approaches were tried and a rich set of experiments performed using resources such as machine translation, parallel corpora, several approaches to stemming and/or morphology, and both pre-translation and post-translation blind relevance feedback.

Overview of Results

50 Citations1997Christian‐D. Schönwiese, Jörg Rapp

Lecture notes in computer scienceThe Domain-Specific Task of CLEF - Specific Evaluation Strategies in Cross-Language Information Retrieval

49 Citations2001Michael Kluck, Fredric C. Gey

The domain-specific cross-language information retrieval (CLIR) task of CLEF is described, why and why it is important and how it differs from general cross- language retrieval problem associated with the general CLEF collections.

Lecture notes in computer scienceThe CLEF 2001 Interactive Track

37 Citations2002Douglas W. Oard, Julio Gonzalo

This paper describes the shared experiment design used at all three participating sites, summarizes preliminary re- sults from the evaluation, and concludes with observations on lessons learned that can inform the design of subsequent evaluation campaigns.

Lecture notes in computer scienceCLEF 2000 — Overview of Results

25 Citations2001Martín Braschler

Details of the various subtasks are presented, and the analysis indicates that the CLEF relevance assessments are of comparable quality to those of the well-known and trusted TREC ad-hoc collections.

Lecture notes in computer scienceCLEF 2001 — Overview of Results

24 Citations2002Martín Braschler

The CLEF multilingual test collection is examined with respect to the completeness of its relevance assessments and indicates that thetest collection is stable and well suited for use in future evaluations.