Few-shot classification in named entity recognition task
Published 8 April 2019Open access
Alexander Fritzler, Varvara Logacheva, Maksim Kretov
Citations188
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work tackles Named Entity Recognition (NER) task using Prototypical Network --- a metric learning technique that learns intermediate representations of words which cluster well into named entity classes.
Abstract
For many natural language processing (NLP) tasks the amount of annotated data is limited. This urges a need to apply semi-supervised learning techniques, such as transfer learning or meta-learning. In this work we tackle Named Entity Recognition (NER) task using Prototypical Network - a metric learning technique. It learns intermediate representations of words which cluster well into named entity classes. This property of the model allows classifying words with extremely limited number of training examples, and can potentially be used as a zero-shot learning method. By coupling this technique with transfer learning we achieve well-performing classifiers trained on only 20 instances of a target class.
Keywords
Computer Science
Glove: Global Vectors for Word Representation
33,769 Citations2014Jeffrey Pennington, Richard Socher +1 more
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
arXiv (Cornell University)Prototypical Networks for Few-shot Learning
5,176 Citations2017Jake Snell, Kevin Swersky +1 more
This work proposes Prototypical Networks for few-shot classification, and provides an analysis showing that some simple design decisions can yield substantial improvements over recent approaches involving complicated architectural choices and meta-learning.
International Conference on Learning RepresentationsOptimization as a Model for Few-Shot Learning
2,440 Citations2017Sachin Ravi, Hugo Larochelle
arXiv (Cornell University)Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition
1,574 Citations2002Erik F. Tjong Kim Sang
arXiv (Cornell University)Matching Networks for One Shot Learning
942 Citations2016Oriol Vinyals, Charles Blundell +3 more
arXiv (Cornell University)A Survey on Metric Learning for Feature Vectors and Structured Data
531 Citations2013Aurélien Bellet, Amaury Habrard +1 more
A systematic review of the metric learning literature is proposed, highlighting the pros and cons of each approach and presenting a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning.
Deep Active Learning for Named Entity Recognition
368 Citations2017Yanyao Shen, Hyokun Yun +3 more
By combining deep learning with active learning, the authors can outperform classical methods even with a significantly smaller amount of training data, and this work shows otherwise.
Institutional Research Information System (Università degli Studi di Trento)Towards Robust Linguistic Analysis using OntoNotes
362 Citations2013Sameer Pradhan, Alessandro Moschitti +6 more
An analysis of the performance of publicly available, state-of-the-art tools on all layers and languages in the OntoNotes v5.0 corpus should set the benchmark for future development of various NLP components in syntax and semantics, and possibly encourage research towards an integrated system that makes use of the various layers jointly to improve overall performance.
arXiv (Cornell University)One-shot Learning with Memory-Augmented Neural Networks
298 Citations2016Adam Santoro, Sergey Bartunov +3 more
The ability of a memory-augmented neural network to rapidly assimilate new data, and leverage this data to make accurate predictions after only a few samples is demonstrated.
Neural Cross-Lingual Named Entity Recognition with Minimal Resources
144 Citations2018Jiateng Xie, Zhilin Yang +3 more
This work proposes a method that finds translations based on bilingual word embeddings and uses self-attention to improve robustness to word order differences, and demonstrates that these methods achieve state-of-the-art or competitive NER performance on commonly tested languages under a cross-lingual setting, with much lower resource requirements than past approaches.
International Conference on Computational LinguisticsLabel Embedding for Zero-shot Fine-grained Named Entity Typing.
118 Citations2016Yukun Ma, Erik Cambria +1 more
A label embedding method that incorporates prototypical and hierarchical information to learn pre-trained label embeddings and a zero-shot learning framework that can predict both seen and previously unseen entity types are presented.
arXiv (Cornell University)SwellShark: A Generative Model for Biomedical Named Entity Recognition without Labeled Data
89 Citations2017Jason Fries, Sen Wu +2 more
This work presents SwellShark, a framework for building biomedical named entity recognition (NER) systems quickly and without hand-labeled data that achieves competitive scores with state-of-the-art supervised benchmarks using no hand- labeled training data.
Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability
85 Citations2017Tiancheng Zhao, Allen Lu +2 more
This framework enables encoder-decoder models to accomplish slot-value independent decision-making and interact with external databases and shows the flexibility of the proposed method by interleaving chatting capability with a slot-filling system for better out-of-domain recovery.
Label-Aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition
80 Citations2018Zhenghui Wang, Yanru Qu +8 more
A label-aware double transfer learning framework (La-DTL) for cross-specialty NER, so that a medical NER system designed for one specialty could be conveniently applied to another one with minimal annotation efforts, is proposed.
arXiv (Cornell University)Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields
65 Citations2024Ryan Cotterell, Kevin Duh
This paper presents a transfer learning scheme, whereby character-level neural CRFs are trained to predict named entities for both high-resource languages and low- resource languages jointly, improving F1 by up to 9.8 points.
arXiv (Cornell University)Improving One-Shot Learning through Fusing Side Information
43 Citations2017Yao-Hung Hubert Tsai, Ruslan Salakhutdinov
This paper introduces two statistical approaches for fusing side information into data representation learning to improve one-shot learning and introduces an attention mechanism to efficiently treat examples belonging to the 'lots-of-examples' classes as quasi-samples (additional training samples) for 'one-example' classes.
Computación y SistemasImproved Named Entity Recognition using Machine Translation-based Cross-lingual Information
14 Citations2016Sandipan Dandapat, Andy Way
A technique to improve named entity recognition in a resource-poor language (Hindi) by using cross-lingual information is described and improves F 1 score by 2.1 points absolute and 2.9% relative over a good-performing baseline model.
Learning How to Self-Learn: Enhancing Self-Training Using Neural Reinforcement Learning
14 Citations2018Chenhua Chen, Yue Zhang +1 more
A deep reinforcement learning method is proposed to learn self-training strategy automatically based on the neural representation of sentences and the hidden features from classifiers, designed to capture their linguistic characteristics and learn an optimal policy for instance selection automatically.
arXiv (Cornell University)Sequence Labeling: A Practical Approach
12 Citations2018Adnan Akhundov, Dietrich Trautmann +1 more
A universal end-to-end Bi-LSTM-based neural sequence labeling model applicable to a wide range of NLP tasks and languages is utilized, which combines morphological, semantic, and structural cues extracted from data to arrive at informed predictions.
arXiv (Cornell University)Learning How to Self-Learn: Enhancing Self-Training Using Neural Reinforcement Learning
2 Citations2018Chenhua Chen, Yue Zhang