Improving NER in Arabic Using a Morphological Tagger.

Published 1 May 2008

B. Farber, Dayne Freitag, Nizar Habash, Owen Rambow

Source URL

Citations43

SJR quartileQ1

SJR score0.48

SNIP1.79

TL;DR

A named entity recognition system for Arabic is discussed, and how MADA, a full morphological tagger which uses a morphological analyzer is incorporated, yielding a 14\% reduction in error over the baseline.

Abstract

We discuss a named entity recognition system for Arabic, and show how we incorporated the information provided by MADA, a full morphological tagger which uses a morphological analyzer. Surprisingly, the relevant features used are the capitalization of the English gloss chosen by the tagger, and the fact that an analysis is returned (that a word is not OOV to the morphological analyzer). The use of the tagger also improves over a third system which just uses a morphological analyzer, yielding a 14 % reduction in error over the baseline. We conduct a thorough error analysis to identify sources of success and failure among the variations, and show that by combining the systems in simple ways we can significantly influence the precision-recall trade-off. 1.

Keywords

Computer Science

Discriminative training methods for hidden Markov models

1,889 Citations2002Michael Collins

Experimental results on part-of-speech tagging and base noun phrase chunking are given, in both cases showing improvements over results for a maximum-entropy tagger.

View PDF

A maximum entropy approach to named entity recognition

467 Citations1999Ralph Grishman, Andrew Borthwick

This thesis describes a novel statistical named-entity recognition system known as MENE (Maximum Entropy Named Entity), and demonstrates the trans-lingual portability of the system, which was competitive with the best systems built by native Japanese speakers despite the fact that the author speaks no Japanese.

Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop

443 Citations2005Nizar Habash, Owen Rambow

An approach to using a morphological analyzer for tokenizing and morphologically tagging Arabic words in one process using classifiers for individual morphological features, as well as ways of using these classifiers to choose among entries from the output of the analyzer.

View PDF

Efficient support vector classifiers for named entity recognition

324 Citations2002Hideki Isozaki, Hideto Kazawa

It is shown that an NE recognizer based on Support Vector Machines (SVMs) gives better scores than conventional systems, but off-the-shelf SVM classifiers are too inefficient for this task.

View PDF

Text, speech and language technologyOn Arabic Transliteration

309 Citations2007Nizar Habash, Abdelhadi Soudi +1 more

This chapter introduces the transliteration scheme used to represent Arabic characters in this book and presents guidelines for Arabic pronunciation using this transliterations scheme.

Name Tagging with Word Clusters and Discriminative Training

270 Citations2004S.L. Miller, Jethran Guinness +1 more

A technique for augmenting annotated training data with hierarchical word clusters that are automatically derived from a large unannotated corpus that achieves a 25% reduction in error over the state-of-the-art HMM trained on the same material.

Text, speech and language technologyArabic Computational Morphology: Knowledge-based and Empirical Methods

198 Citations2007Abdelhadi Soudi, Günter Neumann +1 more

Computational IntelligenceApplying Machine Learning for High‐Performance Named‐Entity Extraction

84 Citations2000Shumeet Baluja, Vibhu O. Mittal +1 more

An extensible system that automatically combines weak evidence from different, easily available sources: parts‐of‐speech tags, dictionaries, and surface‐level syntactic information such as capitalization and punctuation yields a system that achieves performance equivalent to the best existing hand‐crafted approaches.

Text, speech and language technologyArabic Morphological Representations for Machine Translation

75 Citations2007Nizar Habash

Different Arabic morphological representations used by MT-relevant natural language processing resources and tools are described and their usability in different MT approaches are discussed and a common framework for relating different levels of representations to each other is presented.