A workbench for finding structure in texts

Published 1 January 1997Open access

Andrei Mikheev, Steven Finch

Citations52

TL;DR

A set of computational tools with (n)SGML pipeline data flow for uncovering internal structure in natural language texts using the workbench to demonstrate the independence of the text representation and text analysis phases.

Abstract

In this paper we report on a set of compu- tational tools with (n)SGML pipeline data flow for uncovering internal structure in natural language texts. The main idea behind the workbench is the independence of the text representation and text analysis phases. At the representation phase the text is converted from a sequence of characters to features of interest by means of the annotation tools. At the analysis phase those features are used by statistics gathering and inference tools for finding signifi- cant correlations in the texts. The analysis tools are independent of particular assump- tions about the nature of the feature-set and work on the abstract level of feature- elements represented as SGML items.

Keywords

Computer Science

Natural Language EngineeringTechnical terminology: some linguistic properties and an algorithm for identification in text

817 Citations1995John S. Justeson, Slava M. Katz

The paper presents a terminology indentification algorithm that recovers a high proportion of the technical terms in a text, and a high proportaion of the recovered strings are vaild technical terms.

Retrieving collocations from text: Xtract

799 Citations1993Frank Smadja

A set of techniques based on statistical methods for retrieving and identifying collocations from large textual corpora, based on some original filtering methods that allow the production of richer and higher-precision output are described.

The SGML handbook

544 Citations1990Charles F. Goldfarb

This paper introduces generalized markup, a model for generalized markup that automates the very labor-intensive and therefore time-heavy and expensive process of developing and distributing SGML documents.

Computer Speech & LanguageRobust part-of-speech tagging using a hidden Markov model

416 Citations1992Julian Kupiec

A system for part-of-speech tagging is described, based on a hidden Markov model which can be trained using a corpus of untagged text, which results in a model that correctly tags approximately 96% of the text.

Using SGML as a basis for data-intensive NLP

28 Citations1997David McKelvie, C. Brew +1 more

The LT NSL system is described, an architecture for writing corpus processing tools and the advantages and disadvantages of an SGML approach compared with a non-SGML database approach are addressed.

View PDF

Towards a workbench for acquisition of domain knowledge from natural language

21 Citations1995Andrei Mikheev, Steven Finch

The workbench supports an incremental process of corpus analysis starting from a rough automatic extraction and organization of lexico-semantic regularities and ending with a computer supported analysis of extracted data and a semiautomatic refinement of obtained hypotheses.

View PDF