Unified Structure Generation for Universal Information Extraction
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)Published 1 January 2022Open access
Yaojie Lu, Q. Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han
Citations392
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A unified text-to-structure generation framework, namely UIE, which can universally model different IE tasks, adaptively generate targeted structures, and collaboratively learn general IE abilities from different knowledge sources is proposed.
Abstract
Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, Hua Wu. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.
Keywords
Computer Science
UvA-DARE (University of Amsterdam)Adam: A Method for Stochastic Optimization
84,783 Citations2014Diederik P. Kingma, Jimmy Ba
32,525 Citations2019Jacob Devlin, Ming‐Wei Chang +2 more
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanities Use Case
17,334 Citations2019Yinhan Liu, Myle Ott +8 more
This work considers the task of building machine learning models to automatically select the best combination for a problem instance and contributes to the automatic learning of instance features directly from the high-level representation of a problem instance using a transformer encoder.
Neural Architectures for Named Entity Recognition
4,428 Citations2016Guillaume Lample, Miguel Ballesteros +3 more
Comunicacio presentada a la 2016 Conference of the North American Chapter of the Association for Computational Linguistics, celebrada a San Diego (CA, EUA) els dies 12 a 17 of juny 2016.
Proceedings of the AAAI Conference on Artificial IntelligenceConceptNet 5.5: An Open Multilingual Graph of General Knowledge
2,070 Citations2017Robert E. Speer, Joshua Chin +1 more
A new version of the linked open data resource ConceptNet is presented that is particularly well suited to be used with modern NLP techniques such as word embeddings, with state-of-the-art results on intrinsic evaluations of word relatedness that translate into improvements on applications of word vectors, including solving SAT-style analogies.
SemEval-2014 Task 4: Aspect Based Sentiment Analysis
1,881 Citations2014Maria Pontiki, Dimitrios Galanis +4 more
SemEval2014 Task 4 aimed to foster research in the field of aspect-based sentiment analysis, where the goal is to identify the aspects of given target entities and the sentiment expressed for each aspect.
arXiv (Cornell University)Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition
1,574 Citations2002Erik F. Tjong Kim Sang
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
1,572 Citations2021Linting Xue, Noah Constant +6 more
Lecture notes in computer scienceModeling Relations and Their Mentions without Labeled Text
1,331 Citations2010Sebastian Riedel, Limin Yao +1 more
A novel approach to distant supervision that can alleviate the problem of noisy patterns that hurt precision by using a factor graph and applying constraint-driven semi-supervision to train this model without any knowledge about which sentences express the relations in the authors' training KB.
SemEval-2016 Task 5: Aspect Based Sentiment Analysis
1,224 Citations2016Maria Pontiki, Dimitrios Galanis +16 more
This paper describes the SemEval 2016 shared task on Aspect Based Sentiment Analysis (ABSA), a continuation of the respective tasks of 2014 and 2015, which attracted 245 submissions from 29 teams and provided 19 training and 20 testing datasets.
SemEval-2015 Task 12: Aspect Based Sentiment Analysis
1,152 Citations2015Maria Pontiki, Dimitrios Galanis +3 more
The task provided manually annotated reviews in three domains (restaurants, laptops and hotels), and a common evaluation procedure, to foster research beyond sentenceor text-level sentiment classification towards Aspect Based Sentiment Analysis.
arXiv (Cornell University)Sequence Level Training with Recurrent Neural Networks
949 Citations2015Marc’Aurelio Ranzato, Sumit Chopra +2 more
Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme
735 Citations2017Suncong Zheng, Feng Wang +4 more
A novel tagging scheme is proposed that can convert the joint extraction task to a tagging problem, and different end-to-end models are studied to extract entities and their relations directly, without identifying entities and relations separately.
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction
636 Citations2018Yi Luan, Luheng He +2 more
The multi-task setup reduces cascading errors between tasks and leverages cross-sentence relations through coreference links and supports construction of a scientific knowledge graph, which is used to analyze information in scientific literature.
A Unified MRC Framework for Named Entity Recognition
607 Citations2020Xiaoya Li, Jingrong Feng +4 more
This paper proposes to formulate the task of NER as a machine reading comprehension (MRC) task, and naturally tackles the entity overlapping issue in nested NER: the extraction of two overlapping entities with different categories requires answering two independent questions.
arXiv (Cornell University)mT5: A massively multilingual pre-trained text-to-text transformer
601 Citations2020Linting Xue, Noah Constant +6 more
Extracting Relational Facts by an End-to-End Neural Model with Copy Mechanism
518 Citations2018Xiangrong Zeng, Daojian Zeng +3 more
This paper proposes an end-to-end model based on sequence- to-sequence learning with copy mechanism, which can jointly extract relational facts from sentences of any of these classes, including Normal, EntityPairOverlap and SingleEntiyOverlap.
Munich Personal RePEc Archive (Ludwig Maximilian University of Munich)A Linear Programming Formulation for Global Inference in Natural Language Tasks
442 Citations2004Dan Roth, Wen-tau Yih
This work develops a linear programing formulation for this problem and evaluates it in the context of simultaneously learning named entities and relations to efficiently incorporate domain and task specific constraints at decision time, resulting in significant improvements in the accuracy and the "human-like" quality of the inferences.
TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking
438 Citations2020Yucheng Wang, Bowen Yu +4 more
This paper proposes a one-stage joint extraction model, namely, TPLinker, which is capable of discovering overlapping relations sharing one or both entities while being immune from the exposure bias, and formulates joint extraction as a token pair linking problem and introduces a novel handshaking tagging scheme.
A Frustratingly Easy Approach for Entity and Relation Extraction
421 Citations2021Zexuan Zhong, Danqi Chen
This work presents a simple pipelined approach for entity and relation extraction, and establishes the new state-of-the-art on standard benchmarks, obtaining a 1.7%-2.8% absolute improvement in relation F1 over previous joint models with the same pre-trained encoders.
Event Extraction by Answering (Almost) Natural Questions
379 Citations2020Xinya Du, Claire Cardie
This work introduces a new paradigm for event extraction by formulating it as a question answering (QA) task, which extracts the event arguments in an end-to-end manner and outperforms prior methods substantially.
A Joint Neural Model for Information Extraction with Global Features
376 Citations2020Ying Lin, Heng Ji +2 more
A joint neural framework that aims to extract the globally optimal IE result as a graph from an input sentence and can be easily applied to new languages or trained in a multilingual manner, as OneIE does not use any language-specific feature.
Knowledge Base Population: Successful Approaches and Challenges
311 Citations2011Heng Ji, Ralph Grishman
The techniques which can serve as a basis for a good KBP system are provided, the remaining challenges by comparison with traditional Information Extraction (IE) and Question Answering (QA) tasks are laid out, and some suggestions to address these challenges are provided.
Neural Architectures for Nested NER through Linearization
298 Citations2019Jana Straková, Milan Straka +1 more
Two neural network architectures for nested named entity recognition (NER), a setting in which named entities may overlap and also be labeled with more than one label, are proposed and outperform the nested NER state of the art on four corpora.
A Unified Generative Framework for Aspect-based Sentiment Analysis
292 Citations2021Hang Yan, Junqi Dai +3 more
Every subtask target is redefined as a sequence mixed by pointer indexes and sentiment class indexes, which converts all ABSA subtasks into a unified generative formulation and exploits the pre-training sequence-to-sequence model BART to solve all ABsa subtasks in an end- to-end framework.
Template-Based Named Entity Recognition Using BART
290 Citations2021Leyang Cui, Yu Wu +3 more
A template-based method for NER is proposed, treating NER as a language model ranking problem in a sequence-to-sequence framework, where original sentences and statement templates filled by candidate named entity span are regarded as the source sequence and the target sequence, respectively.
Position-Aware Tagging for Aspect Sentiment Triplet Extraction
265 Citations2020Lu Xu, Hao Li +2 more
This work proposes the first end-to-end model with a novel position-aware tagging scheme that is capable of jointly extracting the triplets and results show that jointly capturing elements in the triplet using this approach leads to improved performance over the existing approaches.
A Unified Generative Framework for Various NER Subtasks
257 Citations2021Hang Yan, Tao Gui +4 more
This work proposes to formulate the NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence-to-sequence (Seq2Seq) framework, and can leverage the pre-trained Seq1Seq model to solve all three kinds of N ER subtasks without the special design of the tagging schema or ways to enumerate spans.
Document-Level Event Argument Extraction by Conditional Generation
242 Citations2021Sha Li, Heng Ji +1 more
A document-level neural event argument extraction model is proposed by formulating the task as conditional generation following event templates by creating the first end-to-end zero-shot event extraction framework.
PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction
238 Citations2021Hengyi Zheng, Rui Wen +8 more
A component to predict potential relations is designed, which constrains the following entity extraction to the predicted relation subset rather than all relations; then a relation-specific sequence tagging component is applied to handle the overlapping problem between subjects and objects; finally, a global correspondence component is designed to align the subject and object into a triple with low-complexity.
UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning
234 Citations2021Wei Li, Can Gao +6 more
A UNIfied-MOdal pre-training architecture, namely UNIMO, which can effectively adapt to both single- modal and multi-modal understanding and generation tasks, and is able to learn more generalizable representations.
Deep Exhaustive Model for Nested Named Entity Recognition
233 Citations2018Mohammad Golam Sohrab, Makoto Miwa
The results show that the model outperforms state-of-the-art models on nested and flat NER, achieving 77.1% and 78.4% respectively in terms of F-score, without any external knowledge resources.
Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders
210 Citations2020Jue Wang, Wei Lu
It is argued that it can be beneficial to design two distinct encoders to capture such two different types of information in the learning process, and proposed is the novel {\em table-sequence encoder} where two different encoder -- a table encoder and a sequence encoder are designed to help each other in the representation learning process.
Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction
207 Citations2021Yaojie Lu, Hongyu Lin +7 more
Experimental results show that, by uniformly modeling all tasks in a single model and universally predicting different labels, the Text2Event method can achieve competitive performance using only record-level annotations in both supervised learning and transfer learning settings.
arXiv (Cornell University)Autoregressive Entity Retrieval
200 Citations2020Nicola De Cao, Gautier Izacard +2 more
IEEE Transactions on Neural Networks and Learning SystemsJoint Entity and Relation Extraction With Set Prediction Networks
187 Citations2023Dianbo Sui, Xiangrong Zeng +3 more
This work treats joint entity and relation extraction as a direct set prediction problem, so that the extraction model is not burdened with predicting the order of multiple triples, and proposes networks featured by transformers with non-autoregressive parallel decoding.
Towards Generative Aspect-Based Sentiment Analysis
183 Citations2021Wenxuan Zhang, Xin Li +3 more
This paper proposes to tackle various ABSA tasks in a unified generative framework with two types of paradigms, namely annotation-style and extraction-style modeling, to enable the training process by formulating each ABSA task as a text generation problem.
Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction
167 Citations2021Lu Xu, Yew Ken Chia +1 more
This analysis shows that the proposed span-level approach achieves more significant improvements over the baselines on triplets with multi-word targets or opinions, and distinguishes the opinion and target spans more properly.
arXiv (Cornell University)Unifying Vision-and-Language Tasks via Text Generation
145 Citations2021Jaemin Cho, Jie Lei +2 more
Exploring Sequence-to-Sequence Learning in Aspect Term Extraction
143 Citations2019Dehong Ma, Sujian Li +3 more
To formalize ATE as a sequence-to-sequence (Seq2Seq) learning task where the source sequence and target sequence are composed of words and labels respectively, the proposed gated unit networks and position-aware attention mechanism are designed.
Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning
132 Citations2021Xinyu Wang, Yong Jiang +5 more
This paper finds empirically that the contextual representations computed on the retrieval-based input view, constructed through the concatenation of a sentence and its external contexts, can achieve significantly improved performance compared to the original input view based only on the sentence.
Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks
116 Citations2019Hongyu Lin, Yaojie Lu +2 more
This paper proposes Anchor-Region Networks (ARNs), a sequence-to-nuggets architecture for nested mention detection which first identifies anchor words of all mentions, and then recognizes the mention boundaries for each anchor word by exploiting regular phrase structures.
CLEVE: Contrastive Pre-training for Event Extraction
106 Citations2021Ziqi Wang, Xiaozhi Wang +7 more
CLEVE is proposed, a contrastive pre-training framework for EE to better learn event knowledge from large unsupervised data and their semantic structures (e.g. AMR) obtained with automatic parsers.
Proceedings of the AAAI Conference on Artificial IntelligenceCASIE: Extracting Cybersecurity Event Information from Text
105 Citations2020Taneeya Satyapanich, Francis Ferraro +1 more
CASIE is a system that extracts information about cybersecurity events from text and populates a semantic model that can incorporate rich linguistic features and word embeddings and shows that each subsystem performs well in the event detection pipeline.
Frontiers in artificial intelligence and applicationsJoint Extraction of Entities and Relations Based on a Novel Decomposition Strategy
102 Citations2020Bowen Yu, Zhenyu Zhang +5 more
This paper decomposes the joint extraction task into two interrelated subtasks, namely HE extraction and TER extraction, and deconstructed into several sequence labeling problems based on the proposed span-based tagging scheme, which are conveniently solved by a hierarchical boundary tagger and a multi-span decoding algorithm.
Proceedings of the 2021 Conference on Empirical Methods in Natural Language ProcessingFew-Shot Named Entity Recognition: An Empirical Baseline Study
97 Citations2021Jiaxin Huang, Chunyuan Li +7 more
Three orthogonal schemes to improve model generalization ability in few-shot settings are investigated: meta-learning to construct prototypes for different entity types, task-specific supervised pre-training on noisy web data to extract entity-related representations and self-training to leverage unlabeled in-domain data.
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning
92 Citations2021Yujia Qin, Yankai Lin +7 more
Experimental results demonstrate that ERICA can improve typical PLMs (BERT and RoBERTa) on several language understanding tasks, including relation extraction, entity typing and question answering, especially under low-resource settings.
Nugget Proposal Networks for Chinese Event Detection
84 Citations2018Hongyu Lin, Yaojie Lu +2 more
NPNs are proposed, which can solve the word-trigger mismatch problem by directly proposing entire trigger nuggets centered at each character regardless of word boundaries, and significantly outperform the state-of-the-art methods.
Automatic extraction of facts from press releases to generate news stories
75 Citations1992Peggy M. Andersen, Philip J. Hayes +4 more
JASPER is a fact extraction system recently developed and deployed by Carnegie Group for Reuters Ltd, which uses a template-driven approach, partial understanding techniques, and heuristic procedures to extract certain key pieces of information from a limited range of text.
Natural Language EngineeringTwenty-five years of information extraction
62 Citations2019Ralph Grishman
How information extraction has changed over the past 25 years is described, moving from hand-coded rules to neural networks, with a few stops on the way, to research advances in NLP and to the evaluations organized by the US Government.
Coarse-to-Fine Pre-training for Named Entity Recognition
50 Citations2020Mengge Xue, Bowen Yu +4 more
A NER-specific pre-training framework to in-ject coarse-to-fine automatically mined entity knowledge into pre-trained models and devise a self-supervisedauxiliary task to mine the fine-grained namedentity knowledge via clustering.
Singapore Management University Institutional Knowledge (InK) (Singapore Management University)Generating Templates of Entity Summaries with an Entity-Aspect Model and Pattern Mining
48 Citations2010Peng Li, Jing Jiang +1 more
A novel approach to automatic generation of summary templates from given collections of summary articles by developing an entity-aspect LDA model that automatically grouping of semantically related sentence patterns and automatic identification of template slots that need to be filled in.
Intent Classification and Slot Filling for Privacy Policies
24 Citations2021Wasi Uddin Ahmad, Jianfeng Chi +4 more
This work proposes PolicyIE, an English corpus consisting of 5,250 intent and 11,788 slot annotations spanning 31 privacy policies of websites and mobile applications, and presents two alternative neural approaches as baselines, intent classification and slot filling as a joint sequence tagging and modeling them as a sequence-to-sequence (Seq2Seq) learning task.
Artificial IntelligenceEnd-to-end neural event coreference resolution
20 Citations2021Yaojie Lu, Hongyu Lin +3 more
An End-to-End Event Coreference approach -- E3C neural network, which can jointly model event detection and event coreference resolution tasks, and learn to extract features from raw text automatically, is proposed.
From information retrieval to information extraction
17 Citations2000David Milward, James Thomas
A system which enables users to create on-the-fly queries which involve not just keywords, but also sortal constraints and linguistic constraints, to bridge the gap between keyword based Information Retrieval and pattern based Information Extraction.
HySPA: Hybrid Span Generation for Scalable Text-to-Graph Extraction
15 Citations2021Liliang Ren, Chenkai Sun +2 more
A Hybrid Span Generator (HySPA) is proposed that invertibly maps the information graph to an alternating sequence of nodes and edge types, and directly generates such sequences via a hybrid span decoder which can decode both the spans and the types recurrently in linear time and space complexities.
Issues and methodology for template design for information extraction
4 Citations1994Boyan Onyshkevych
This paper discusses template structure and methodological issues which arise in the template design process, within the context of a discussion of the design process itself, based on the templateDesign process for TIPSTER/MUC5 and certain subsequent Information Extraction exercises.