Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the Web

Published 1 August 1999

Jian‐Yun Nie, Michel Simard, Pierre Isabelle, Richard Durand

Citations310

TL;DR

It is shown that using a probabilistic model, it is able to obtain performances close to those using an MT system, and the possibility of automatically gather parallel texts from the Web in an attempt to construct a reasonable training corpus is investigated.

Abstract

This paper describes the use of a probabilistic translation model to cross-language IR (CLIR). The performance of this approach is compared with that using machine translation (MT). It is shown that using a probabilistic model, we are able to obtain performances close to those using an MT system. In addition, we also investigated the possibility of automatically gather parallel texts from the Web in an attempt to construct a reasonable training corpus. The result is very encouraging. We showed that in several tests, such a training corpus is as good as a manually constructed one for CLIR purposes.

Keywords

Computer Science

The mathematics of statistical machine translation: parameter estimation

4,125 Citations1993Peter F. Brown, Vincent J. Della Pietra +2 more

It is reasonable to argue that word-by-word alignments are inherent in any sufficiently large bilingual corpus, given a set of pairs of sentences that are translations of one another.

A program for aligning sentences in bilingual corpora

956 Citations1993William A. Gale, Kenneth Church

This paper will describe a method and a program for aligning sentences based on a simple statistical model of character lengths, which uses the fact that longer sentences in one language tend to be translated into longer sentence in the other language, and that shorter sentences tend to been translated into shorter sentences.

Resolving ambiguity for cross-language retrieval

310 Citations1998Lisa Ballesteros, W. Bruce Croft

A technique based on co-occurrence statistics from unlinked corpora which can be used to reduce the ambiguity associated with phrasal and term translation is presented and combined with other techniques for reducing ambiguity is combined to achieve more than 90% monolingual effectiveness.

View PDF

Cross-Language Information Retrieval

226 Citations1998Gregory Grefenstette

This work focuses on the development of a model for automatic Cross-Language Information Retrieval using Latent Semantic Indexing and its application to Machine Translation Technology.

Machine TranslationTarget-Text Mediated Interactive Machine Translation

123 Citations1997George Foster, Pierre Isabelle +1 more

This work describes the design, implementation, and performance of an automatic word completion system for translators which is intended to demonstrate the feasibility of the proposed interactive machine translation (IMT) approach, albeit in a very rudimentary form.

Lecture notes in computer scienceParallel Strands: A Preliminary Investigation into Mining the Web for Bilingual Text

99 Citations1998Philip Resnik

A method for automatically finding parallel translated documents on the Web is presented, conceptually simple, fully language independent, and scalable, and preliminary evaluation results indicate that the method may be accurate enough to apply without human intervention.

Artificial IntelligenceTranslingual information retrieval: learning from bilingual corpora

96 Citations1998Yiming Yang, Jaime Carbonell +2 more

The results show that using bilingual corpora for automated extraction of term equivalences in context outperforms dictionarybased methods and is comparable to that of other statistical corpus-based methods.

QUILT

66 Citations1997Mark M. Davis, William C. Ogden

Q~T (QWXYUaa Interfacewith Light Tmnaladon8)k a prototype implementadon of a completecmas-langusge textretrieval 8ystemtbat takeaFkgli8hqude3snd pmduceaw811glo8a translations of Spani8hdocumenla.

View PDF

Using a Probabilistic Translation Model for Cross-Language Information Retrieval

15 Citations1998Jian‐Yun Nie, Pierre Isabelle +1 more

The results indicate that translating queries with off-theshelf machine translation systems can result in relatively good performance, but the results also indicate that other methods can perfonn even better.