Neural Named Entity Recognition Using a Self-Attention Mechanism

A novel supervised approach for text tagging and multi-label text classification based on a multi-head encoder-decoder neural network architecture that achieves a significantly better performance than baselines with an equivalent number of parameters is proposed.

Abstract

We propose a novel supervised approach for text tagging and multi-label text classification based on a multi-head encoder-decoder neural network architecture. Our method predicts which subset of possible tags best matches an input text. It efficiently spends computational resources, exploiting dependencies between tags by encoding an input text into a compact representation which is then passed to multiple decoder classifier heads. We test our architecture on a Twitter hashtag prediction task, comparing it to a baseline model with multiple feedforward networks and a baseline model with multiple recurrent neural networks with GRU cells. We show that our approach achieves a significantly better performance than baselines with an equivalent number of parameters.