Distilling Discrimination and Generalization Knowledge for Event Detection via Delta-Representation Learning
Published 1 January 2019Open access
Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
Citations44
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A Delta-learning approach to distill discrimination and generalization knowledge is proposed by effectively decoupling, incrementally learning and adaptively fusing event representation, and achieves state-of-the-art performance on both ACE2005 and KBP2017 datasets.
Abstract
Event detection systems rely on discrimination knowledge to distinguish ambiguous trigger words and generalization knowledge to detect unseen/sparse trigger words. Current neural event detection approaches focus on trigger-centric representations, which work well on distilling discrimination knowledge, but poorly on learning generalization knowledge. To address this problem, this paper proposes a -learning approach to distill discrimination and generalization knowledge by effectively decoupling, incrementally learning and adaptively fusing event representation. Experiments show that our method significantly outperforms previous approaches on unseen/sparse trigger words, and achieves state-of-the-art performance on both ACE2005 and KBP2017 datasets.
Keywords
Computer Science
Journal of Japan Society for Fuzzy Theory and Intelligent InformaticsGAN（Generative Adversarial Nets）
21,715 Citations2017Ian Goodfellow
Generative Adversarial Nets
19,938 Citations2023Raphael Labaca-Castro
Advances in computer vision and pattern recognitionDomain-Adversarial Training of Neural Networks
7,498 Citations2017Yaroslav Ganin, Evgeniya Ustinova +6 more
A new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions, which can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer.
arXiv (Cornell University)ADADELTA: An Adaptive Learning Rate Method
5,531 Citations2012Matthew D. Zeiler
A novel per-dimension learning rate method for gradient descent called ADADELTA that dynamically adapts over time using only first order information and has minimal computational overhead beyond vanilla stochastic gradient descent is presented.
Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks
929 Citations2015Yubo Chen, Liheng Xu +3 more
A word-representation model to capture meaningful semantic regularities for words and a framework based on a convolutional neural network to capture sentence-level clues are introduced.
Topic detection and tracking: event-based information organization
682 Citations2002James Allan
This chapter defines the basic concepts of TDT and provides historical context for the concepts and provides an overview of the technical approaches that have been used and that have succeeded.
Joint Event Extraction via Recurrent Neural Networks
665 Citations2016Thien Huu Nguyen, Kyunghyun Cho +1 more
This work proposes to do event extraction in a joint framework with bidirectional recurrent neural networks, thereby benefiting from the advantages of the two models as well as addressing issues inherent in the existing approaches.
Unsupervised Learning of Narrative Event Chains
565 Citations2008Nathanael Chambers, Dan Jurafsky
A three step process to learning narrative event chains using unsupervised distributional methods to learn narrative relations between events sharing coreferring arguments and introduces two evaluations: the narrative cloze to evaluate event relatedness, and an order coherence task to evaluate narrative order.
The stages of event extraction
511 Citations2006David Ahn
This paper presents a simple, modular approach to event extraction that allows for experiment with a variety of machine learning methods for sub-tasks of varying difficulty, as well as to evaluate the impact on performance these sub-Tasks have on the overall task.
Joint Event Extraction via Structured Prediction with Global Features
478 Citations2013Qi Li, Heng Ji +1 more
This work proposes a joint framework based on structured prediction which extracts triggers and arguments together so that the local predictions can be mutually improved, and proposes to incorporate global features which explicitly capture the dependencies of multiple triggers and argued.
Event Detection and Domain Adaptation with Convolutional Neural Networks
416 Citations2015Thien Huu Nguyen, Ralph Grishman
Convolutional neural networks that overcome the two fundamental limitations of the traditional feature-based approaches to this task: complicated feature engineering for rich feature sets and error propagation from the preceding stages which generate these features are studied.
Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation
398 Citations2018Xiao Liu, Zhunchen Luo +1 more
This paper proposes a novel Jointly Multiple Events Extraction framework to jointly extract multiple event triggers and arguments by introducing syntactic shortcut arcs to enhance information flow and attention-based graph convolution networks to model graph information.
Proceedings of the AAAI Conference on Artificial IntelligenceGraph Convolutional Networks With Argument-Aware Pooling for Event Detection
386 Citations2018Thien Huu Nguyen, Ralph Grishman
This work investigates a convolutional neural network based on dependency trees to perform event detection and proposes a novel pooling method that relies on entity mentions to aggregate the convolution vectors.
Using Document Level Cross-Event Inference to Improve Event Extraction
296 Citations2010Shasha Liao, Ralph Grishman
This paper uses document level information from the training corpus to improve the performance of ACE event extraction and uses information about other types of events to make predictions or resolve ambiguities regarding a given event.
Leveraging Knowledge Bases in LSTMs for Improving Machine Reading
258 Citations2017Bishan Yang, Tom M. Mitchell
KBLSTM is proposed, a novel neural model that leverages continuous representations of KBs to enhance the learning of recurrent neural networks for machine reading and achieves accuracies that surpass the previous state-of-the-art results for both entity extraction and event extraction on the widely used ACE2005 dataset.
Proceedings of the AAAI Conference on Artificial IntelligenceJointly Extracting Event Triggers and Arguments by Dependency-Bridge RNN and Tensor-Based Argument Interaction
255 Citations2018Lei Sha, Feng Qian +2 more
A novel dependency bridge recurrent neural network (dbRNN) is proposed that simultaneously applying tree structure and sequence structure in RNN brings much better performance than only uses sequential RNN.
Exploiting Argument Information to Improve Event Detection via Supervised Attention Mechanisms
249 Citations2017Shulin Liu, Yubo Chen +2 more
This work systematically investigates the proposed model under the supervision of different attention strategies and shows that the approach advances state-of-the-arts and achieves the best F1 score on ACE 2005 dataset.
Adversarial Removal of Demographic Attributes from Text Data
245 Citations2018Yanai Elazar, Yoav Goldberg
It is shown that demographic information of authors is encoded in—and can be recovered from—the intermediate representations learned by text-based neural classifiers, and the implication is that decisions of classifiers trained on textual data are not agnostic to—and likely condition on—demographic attributes.
Automatically Labeled Data Generation for Large Scale Event Extraction
172 Citations2017Yubo Chen, Shulin Liu +3 more
This work proposes to automatically label training data for event extraction via world knowledge and linguistic knowledge, which can detect key arguments and trigger words for each event type and employ them to label events in texts automatically.
A Language-Independent Neural Network for Event Detection
146 Citations2016Xiaocheng Feng, Lifu Huang +4 more
Proceedings of the AAAI Conference on Artificial IntelligenceEvent Detection via Gated Multilingual Attention Mechanism
127 Citations2018Jian Liu, Yubo Chen +2 more
A novel multilingual approach---dubbed as Gated Multilingual Attention (GMLATT) framework---to address the two issues simultaneously of data scarcity and monolingual ambiguity, which significantly outperforms state-of-the-art methods.
Leveraging FrameNet to Improve Automatic Event Detection
126 Citations2016Shulin Liu, Yubo Chen +3 more
This paper proposes a global inference approach to detect events in FrameNet and analyzes possible mappings from frames to event-types, and improves the performance of event detection.
Modeling Skip-Grams for Event Detection with Convolutional Neural Networks
123 Citations2016Thien Huu Nguyen, Ralph Grishman
A systematic evaluation on both the general setting and the domain adaptation setting demonstrates the effectiveness of the non-consecutive CNN model, leading to the significant performance improvement over the current state-of-the-art systems.
Collective Event Detection via a Hierarchical and Bias Tagging Networks with Gated Multi-level Attention Mechanisms
114 Citations2018Yubo Chen, Hang Yang +3 more
A hierachical and bias tagging networks to detect multiple events in one sentence collectively and a gated multi-level attention to automatically extract and dynamically fuse the sentence-level and document-level information to solve the two problems simultaneously.
Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification
88 Citations2017Lianhui Qin, Zhisong Zhang +3 more
This work develops an adversarial model to enable an adaptive imitation scheme through competition between the implicit network and a rival feature discriminator, and achieves state-of-the-art performance on the PDTB benchmark.
Nugget Proposal Networks for Chinese Event Detection
84 Citations2018Hongyu Lin, Yaojie Lu +2 more
NPNs are proposed, which can solve the word-trigger mismatch problem by directly proposing entire trigger nuggets centered at each character regardless of word boundaries, and significantly outperform the state-of-the-art methods.
Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination
72 Citations2018Jiali Zeng, Jinsong Su +5 more
This paper jointly model NMT with monolingual attention-based domain classification tasks and improves NMT as follows, distinguishing and exploiting word-level domain contexts for multi-domain NMT.
Proceedings of the AAAI Conference on Artificial IntelligenceExploiting the Ground-Truth: An Adversarial Imitation Based Knowledge Distillation Approach for Event Detection
70 Citations2019Jian Liu, Yubo Chen +1 more
This paper proposes an adversarial imitation based knowledge distillation approach, for the first time, to tackle the challenge of acquiring knowledge from rawsentences for event detection.
Proceedings of the AAAI Conference on Artificial IntelligenceA Probabilistic Soft Logic Based Approach to Exploiting Latent and Global Information in Event Classification
66 Citations2016Shulin Liu, Kang Liu +2 more
A feasible approach which encodes global information in the form of logic using Probabilistic Soft Logic model is proposed, which advances state-of-the-art methods, and achieves the best F1 score to date on the ACE data set.
Self-regulation: Employing a Generative Adversarial Network to Improve Event Detection
66 Citations2018Yu Hong, Wenxuan Zhou +3 more
A self-regulated learning approach is proposed by utilizing a generative adversarial network to generate spurious features and employing a recurrent network to eliminate the fakes.
International Joint Conference on Natural Language ProcessingExploiting Document Level Information to Improve Event Detection via Recurrent Neural Networks
55 Citations2017Shaoyang Duan, Ruifang He +1 more
A novel document level Recurrent Neural Networks (DLRNN) model is proposed, which can automatically extract cross-sentence clues to improve sentence level event detection without designing complex reasoning rules and outperforms other state-of-the-art methods on ACE 2005 dataset without external knowledge base.
Semi-Supervised Event Extraction with Paraphrase Clusters
40 Citations2018James F. Ferguson, Colin Lockard +2 more
This work presents a method for self-training event extraction systems by bootstrapping additional training data by taking advantage of the occurrence of multiple mentions of the same event instances across newswire articles from multiple sources.
Event Detection with Neural Networks: A Rigorous Empirical Evaluation
34 Citations2018Walker Orr, Prasad Tadepalli +1 more
A novel GRU-based model that combines syntactic information along with temporal structure through an attention mechanism is presented that is competitive with other neural network architectures through empirical evaluations under different random initializations and training-validation-test splits of ACE2005 dataset.
Exploiting Contextual Information via Dynamic Memory Network for Event Detection
31 Citations2018Shaobo Liu, Rui Cheng +2 more
This work proposes the trigger detection dynamic memory network (TD-DMN), arguing that the context can be better exploited by processing the context multiple times, allowing the model to perform complex reasoning and to generate better context representation, thus improving the overall performance.
Similar but not the Same: Word Sense Disambiguation Improves Event Detection via Neural Representation Matching
31 Citations2018Weiyi Lu, Thien Huu Nguyen
A method to transfer the knowledge learned on WSD to ED by matching the neural representations learned for the two tasks is proposed and demonstrated to be effective.
Adaptive Scaling for Sparse Detection in Information Extraction
13 Citations2018Hongyu Lin, Yaojie Lu +2 more
This paper proposes adaptive scaling, an algorithm which can handle the positive sparsity problem and directly optimize over F-measure via dynamic cost-sensitive learning and borrows the idea of marginal utility from economics to propose a theoretical framework for instance importance measuring.