Hungarian and Czech Stemming using YASS.

CLEF (Working Notes)Published 1 January 2007

Prasenjit Majumder, Mandar Mitra, Dipasree Pal

Citations8

TL;DR

It is found that, for both languages, the performance of the statistical stemmer is comparable to that of an available rule-based stemmer.

Abstract

This is the second year in a row we are participating in CLEF. Our aim is to test the performance of a statistical stemmer on various languages. Last year, we tried the stemmer on French; this year, we opted for Hungarian, Bulgarian and Czech. We were unable to complete the Bulgarian task, but submitted official runs for the adhoc monolingual Hungarian and Czech tasks. We find that, for both languages, the performance of the statistical stemmer is comparable to that of an available rule-based stemmer.

Keywords

Computer Science

Text REtrieval ConferenceUsing Query Zoning and Correlation Within SMART: TREC 5.

64 Citations1996Chris Buckley, Amit Singhal +1 more

The major focus this year is on zoning different parts of an initial retrieval ranking, and treating each type of query zone differently as processing continues, as well as experiment with dynamic phrasing.

Lecture notes in computer scienceStemming Approaches for East European Languages

30 Citations2008Ljiljana Dolamic, Jacques Savoy

This CLEF evaluation campaign proposes and evaluates various indexing and search strategies for the Czech language that will hopefully result in more effective retrieval than language-independent approaches (n-gram) and proposes a light stemmer and also a second one based on a more aggressive suffix-stripping scheme that will remove some derivational suffixes.

Information Processing & ManagementSearching strategies for the Hungarian language

27 Citations2007Jacques Savoy

It is demonstrated that applying an automatic decompounding procedure for both queries and documents significantly improves IR performance (+10%), compared to word-based indexing strategies.

Lecture notes in computer scienceReport on CLEF-2003 Monolingual Tracks: Fusion of Probabilistic Models for Effective Monolingual Retrieval

26 Citations2004Jacques Savoy

This paper proposes more effective and general stopword lists for the Swedish, Finnish and Russian languages, along with an improved, more efficient and simpler stemming procedure for these three languages, and suggests a combined search approach based on a data fusion strategy that would work with various European languages.

Lecture notes in computer scienceFour Stemmers and a Funeral: Stemming in Hungarian at CLEF 2005

12 Citations2006Anna Tordai, Maarten de Rijke

It is found that stemming over a broad range of suffixes especially on nouns is highly useful and therefore the ad-hoc monolingual task for CLEF 2005.

Lecture notes in computer scienceStatistical vs. Rule-Based Stemming for Monolingual French Retrieval

7 Citations2007Prasenjit Majumder, Mandar Mitra +1 more

The results show that stemming significantly improves retrieval performance (as expected) by about 9-10%, and the performance of the statistical stemmer is comparable with that of the rule-based stemmer.