Fast Semantic Extraction Using a Novel Neural Network Architecture

Abstract

Semantic understanding of text plays an important role in many fashionable tasks such as information (e.g. opinion) extraction, in web-crawling systems, question and answering based systems (e.g. for call centers), as well as in machine translation, summarization and search. End user applications typically have to deal with an enormous quantity of data, e.g. web based systems process large numbers of documents, and interactive human-machine applications require almost instant response. While complete semantic understanding is still a far-distant goal of natural language processing, several intermediate processing tasks are useful for the tasks described above. These range from syntactic analysis (such as part-of-speech labeling and parsing) to semantic analysis (word-sense disambiguation, semantic-role labeling, named entity extraction, co-reference resolution and entailment). Unfortunately, the state-of-the-art solutions of many of these tasks are simply too slow for the applications previously described. For example, state-of-the-art syntactic parsers typically have cubic complexity in the sentence length [2], and several semantic extraction algorithms use the parse-tree as an initial feature. In this work, we focus on the semantic role labeling problem: being able to give a semantic role to a syntactic consituent of a sentence. (However, our results could extend to other domains as