Joint entity and relation extraction based on a hybrid neural network
NeurocomputingPublished 14 February 2017
Suncong Zheng, Yuexing Hao, Dongyuan Lü, Hongyun Bao, Jiaming Xu, Hongwei Hao
Citations257
SJR quartileQ1
SJR score1.47
SNIP1.94
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A hybrid neural network model is proposed to extract entities and their relationships without any handcrafted features to achieve the state-of-the-art results on entity and relation extraction task.
Abstract
Entity and relation extraction is a task that combines detecting entity mentions and recognizing entities' semantic relationships from unstructured text. We propose a hybrid neural network model to extract entities and their relationships without any handcrafted features. The hybrid neural network contains a novel bidirectional encoder-decoder LSTM module (BiLSTM-ED) for entity extraction and a CNN module for relation classification. The contextual information of entities obtained in BiLSTM-ED further pass though to CNN module to improve the relation classification. We conduct experiments on the public dataset ACE05 (Automatic Content Extraction program) to verify the effectiveness of our method. The method we proposed achieves the state-of-the-art results on entity and relation extraction task.
Keywords
Computer Science
Proceedings of the IEEEGradient-based learning applied to document recognition
58,219 Citations1998Yann LeCun, Léon Bottou +2 more
This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.
Ecological ModellingMaximum entropy modeling of species geographic distributions
17,674 Citations2005Steven J. Phillips, Robert P. Anderson +1 more
The maximum entropy method (Maxent) is introduced for modeling species geographic distributions with presence-only data and compared with a commonly used presence- only modeling method, the Genetic Algorithm for Rule-Set Prediction (GARP).
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
arXiv (Cornell University)Natural Language Processing (almost) from Scratch
5,172 Citations2011Ronan Collobert, Jason Weston +4 more
Neural Architectures for Named Entity Recognition
4,428 Citations2016Guillaume Lample, Miguel Ballesteros +3 more
Comunicacio presentada a la 2016 Conference of the North American Chapter of the Association for Computational Linguistics, celebrada a San Diego (CA, EUA) els dies 12 a 17 of juny 2016.
Natural Language Processing (almost) from Scratch
3,987 Citations2011Ronan Collobert, Jason Weston +4 more
Leibniz-Zentrum für Informatik (Schloss Dagstuhl)NER in Archival Finding Aids
3,270 Citations2021Zhiheng Huang, Wei Xu +1 more
This work has created a set of datasets to train Machine Learning algorithms to find named entities and geographical locations and tested them in several datasets and registered their precision and accuracy.
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
2,589 Citations2016Xuezhe Ma, Eduard Hovy
A novel neutral network architecture is introduced that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF, thus making it applicable to a wide range of sequence labeling tasks.
Lingvisticae InvestigationesA survey of named entity recognition and classification
2,488 Citations2007David R. Nadeau, Satoshi Sekine
Observations about languages, named entity types, domains and textual genres studied in the literature, along with other critical aspects of NERC such as features and evaluation methods, are reported.
Design challenges and misconceptions in named entity recognition
1,476 Citations2009Lev Ratinov, Dan Roth
Some of the fundamental design challenges and misconceptions that underlie the development of an efficient and robust NER system are analyzed, and several solutions to these challenges are developed.
Relation Classification via Convolutional Deep Neural Network
1,412 Citations2014Daojian Zeng, Kang Liu +3 more
This paper exploits a convolutional deep neural network (DNN) to extract lexical and sentence level features from the output of pre-existing natural language processing systems and significantly outperforms the state-of-the-art methods.
Semantic Compositionality through Recursive Matrix-Vector Spaces
1,297 Citations2012Richard Socher, Brody Huval +2 more
A recursive neural network model that learns compositional vector representations for phrases and sentences of arbitrary syntactic type and length and can learn the meaning of operators in propositional logic and natural language is introduced.
Improving deep neural networks for LVCSR using rectified linear units and dropout
1,270 Citations2013George E. Dahl, Tara N. Sainath +1 more
Modelling deep neural networks with rectified linear unit (ReLU) non-linearities with minimal human hyper-parameter tuning on a 50-hour English Broadcast News task shows an 4.2% relative improvement over a DNN trained with sigmoid units, and a 14.4% relative improved over a strong GMM/HMM system.
arXiv (Cornell University)Convolutional Neural Networks for Sentence Classification
1,116 Citations2014Yoon Kim
Proceedings of the 22nd international conference on Machine learning - ICML '05
839 Citations2005
This volume, which is also available online from http://www.machinelearning.org, contains the papers accepted for presentation at ICML-2005, the 22nd lnternational Conference on Machine Learning, which was held at the University of Bonn in Germany from August 7 to August 11, 2005.
Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Paths
675 Citations2015Yan Xu, Lili Mou +4 more
This paper presents SDP-LSTM, a novel neural network to classify the relation of two entities in a sentence, which leverages the shortest dependency path (SDP) between two entities; multichannel recurrent neural networks, with long short term memory (L STM) units, pick up heterogeneous information along the SDP.
Addressing the Rare Word Problem in Neural Machine Translation
660 Citations2015Thang Luong, Ilya Sutskever +3 more
This paper proposes and implements an effective technique to address the problem of end-to-end neural machine translation's inability to correctly translate very rare words, and is the first to surpass the best result achieved on a WMT’14 contest task.
Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations
608 Citations2004Nanda Kambhatla
This work employs Maximum Entropy models to combine diverse lexical, syntactic and semantic features derived from the text to obtain competitive results in the Automatic Content Extraction (ACE) evaluation.
Incremental Joint Extraction of Entity Mentions and Relations
553 Citations2014Qi Li, Heng Ji
An incremental joint framework to simultaneously extract entity mentions and relations using structured perceptron with efficient beam-search is presented, which significantly outperforms a strong pipelined baseline, which attains better performance than the best-reported end-to-end system.
Studies in computational intelligenceSupervised Sequence Labelling
547 Citations2012Alex Graves
This chapter provides the background material and literature review for supervised sequence labelling, and describes the different classes of sequencelabelling task that arise under different assumptions about the label sequences.
Classifying Relations by Ranking with Convolutional Neural Networks
517 Citations2015Cícero dos Santos, Bing Xiang +1 more
This work proposes a new pairwise ranking loss function that makes it easy to reduce the impact of artificial classes and shows that it is more effective than CNN followed by a softmax classifier and using only word embeddings as input features is enough to achieve state-of-the-art results.
arXiv (Cornell University)A Convolutional Neural Network for Modelling Sentences
482 Citations2014Nal Kalchbrenner, Edward Grefenstette +1 more
Modeling Joint Entity and Relation Extraction with Table Representation
388 Citations2014Makoto Miwa, Yutaka Sasaki
The experimental results demonstrate that a joint learning approach significantly outperforms a pipeline approach by incorporating global features and by selecting appropriate learning methods and search orders.
IEEE Transactions on CyberneticsDeep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking
371 Citations2016Jun Yu, Xiaokang Yang +2 more
This paper develops a novel deep multimodal distance metric learning (Deep-MDML) method, which adopts a new ranking model to use multi-modal features, including click features and visual features in DML.
IEEE Transactions on Information Forensics and SecurityiPrivacy: Image Privacy Protection by Identifying Sensitive Objects via Deep Multi-Task Learning
344 Citations2016Jun Yu, Baopeng Zhang +3 more
This paper consists of the following contributions: massive social images and their privacy settings are leveraged to learn the object-privacy relatedness effectively and identify a set of privacy-sensitive object classes automatically and a deep multi-task learning algorithm is developed.
NeurocomputingSemantic expansion using word embedding clustering and convolutional neural network for improving short text classification
305 Citations2015Peng Wang, Bo Xu +4 more
A unified framework to expand short texts based on word embedding clustering and convolutional neural network and semantic cliques via fast clustering is proposed, which validates the effectiveness of the proposed method on two open benchmarks.
Proceedings of the 19th ACM international conference on Multimedia
304 Citations2011
Bidirectional Long Short-Term Memory Networks for Relation Classification
304 Citations2015Shu Zhang, Dequan Zheng +2 more
Bidirectional long short-term memory networks (BLSTM) is proposed to model the sentence with complete, sequential information about all words, and features derived from the lexical resources such as WordNet or NLP systems such as dependency parser and named entity recognizers are used.
arXiv (Cornell University)Relation Classification via Recurrent Neural Network
283 Citations2015Dongxu Zhang, Dong Wang
Experiments strongly indicates that the RNN-based model can deliver better performance on relation classification, and it is particularly capable of learning long-distance relation patterns, which makes it suitable for real-world applications where complicated expressions are often involved.
Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling
275 Citations2015Kun Xu, Yansong Feng +2 more
This paper proposes to learn more robust relation representations from shortest dependency paths through a convolution neural network, takes the relation directionality into account and proposes a straightforward negative sampling strategy to improve the assignment of subjects and objects.
National Research Council Canada (Government of Canada)Long Short-Term Memory Over Recursive Structures
272 Citations2015Xiaodan Zhu, Parinaz Sobihani +1 more
This paper proposes to extend chain-structured long short-term memory to tree structures, in which a memory cell can reflect the history memories of multiple child cells or multiple descendant cells in a recursive process, and calls the model S-LSTM, which provides a principled way of considering long-distance interaction over hierarchies.
Joint Entity Recognition and Disambiguation
265 Citations2015Gang Luo, Xiaojiang Huang +2 more
JERL is the first model to jointly optimize NER and linking tasks together completely, and in experiments on the CoNLL’03/AIDA data set, JERL outperforms state-of-art NER and linking systems.
Natural language question answering over RDF
236 Citations2014Lei Zou, Ruizhe Huang +4 more
A semantic query graph is proposed to model the query intention in the natural language question in a structural way, based on which, RDF Q/A is reduced to subgraph matching problem and resolves the ambiguity of natural language questions at the time when matches of query are found.
UTD: Classifying Semantic Relations by Combining Lexical and Semantic Resources
219 Citations2010Bryan Rink, Sanda M. Harabagiu
This paper describes the system for SemEval-2010 Task 8 on multi-way classification of semantic relations between nominals using SVM classifiers and a number of features that capture the context, semantic role affiliation, and possible pre-existing relations of the nominals.
Joint Inference for Fine-grained Opinion Extraction
198 Citations2013Bishan Yang, Claire Cardie
Experimental results demonstrate that the joint inference approach significantly outperforms traditional pipeline methods and baselines that tackle subtasks in isolation for the problem of opinion extraction.
arXiv (Cornell University)End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures
197 Citations2016Makoto Miwa, Mohit Bansal
Exploiting Syntactico-Semantic Structures for Relation Extraction
193 Citations2011Yee Seng Chan, Dan Roth
A novel algorithmic approach to RE is proposed that starts by first identifying structures and then, within these, identifying the semantic type of the relation, which provides significant improvement in RE performance.
Lecture notes in computer scienceMulti-category Classification by Soft-Max Combination of Binary Classifiers
164 Citations2003Kai-Bo Duan, S. Sathiya Keerthi +3 more
A multi-category classification method that combines binary classifiers through soft-max function that is competitive with other implementations of one-versus-all and one-Versus-one methods in terms of both classification accuracy and posteriori probability estimate.
The MIT Press eBooksGlobal Inference for Entity and Relation Identification via a Linear Programming Formulation
150 Citations2007Dan Roth, Wen-tau Yih
It is shown that global inference improves stand-alone learning and allows us to efficiently incorporate expressive domain and task specific constraints at decision time, resulting, beyond significant improvements in the accuracy, in “coherent” quality of the inference.
Convolution kernels on constituent, dependency and sequential structures for relation extraction
118 Citations2009Truc-Vien T. Nguyen, Alessandro Moschitti +1 more
This paper explores the use of innovative kernels based on syntactic and semantic structures for a target relation extraction task and illustrates that the combination of the above kernels achieves high effectiveness and significantly improves the current state-of-the-art.
Transactions of the Association for Computational LinguisticsNamed Entity Recognition with Bidirectional LSTM-CNNs
115 Citations2016Jason Chiu, Eric Nichols
A novel neural network architecture is presented that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering.
arXiv (Cornell University)Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path
114 Citations2015Yan Xu, Lili Mou +4 more
IEEE Transactions on MultimediaUser-Aware Image Tag Refinement via Ternary Semantic Analysis
113 Citations2012Jitao Sang, Changsheng Xu +1 more
This work proposes a method of Ranking based Multi-correlation Tensor Factorization (RMTF), to jointly model the ternary relations among user, image, and tag, and further to precisely reconstruct the user-aware image-tag associations as a result.
arXiv (Cornell University)Classifying Relations by Ranking with Convolutional Neural Networks
113 Citations2015Cícero Nogueira dos Santos, Bing Xiang +1 more
Jointly Identifying Entities and Extracting Relations in Encyclopedia Text via A Graphical Model Approach
104 Citations2010Xiaofeng Yu, Wai Lam
A joint discriminative probabilistic model with arbitrary graphical structure to optimize all relevant subtasks simultaneously and a new inference method, namely collective iterative classification (CIC), to find the most likely assignments for both entities and relations are proposed.
Joint inference of entities, relations, and coreference
101 Citations2013Sameer Singh, Sebastian Riedel +3 more
A single, joint graphical model is proposed that represents the various dependencies between the tasks, allowing flow of uncertainty across task boundaries, and presents a novel extension to belief propagation that sparsifies the domains of variables during inference.
NeurocomputingAn empirical convolutional neural network approach for semantic relation classification
87 Citations2016Pengda Qin, Weiran Xu +1 more
This work deals with the relation classification task utilizing a convolutional neural network approach to automatically control feature learning from raw sentences and minimize the application of external toolkits and resources and shows that the proposed architecture significantly outperforms the state-of-the-art systems.
Lexicon Infused Phrase Embeddings for Named Entity Resolution
68 Citations2014Alexandre Passos, Vineet Kumar +1 more
A new form of learning word embeddings that can leverage information from relevant lexicons to improve the representations, and the first system to use neural word embedDings to achieve state-of-the-art results on named-entity recognition in both CoNLL and Ontonotes NER are presented.
Right buddy makes the difference
65 Citations2012Jitao Sang, Changsheng Xu
Qualitative as well as quantitative evaluation results have validated the effectiveness of the topic-sensitive influencer mining model, and demonstrated the advantage of incorporating topic- sensitive influence in personalized image search and topic-based image recommendation.
Chain Based RNN for Relation Classification
62 Citations2015Javid Ebrahimi, Dejing Dou
A novel approach for relation classification, using a recursive neural network (RNN), based on the shortest path between two entities in a dependency graph, called chains, which performs considerably faster and achieves better classification results.
Knowledge-Based SystemsA neural network framework for relation extraction: Learning entity semantic and relation pattern
60 Citations2016Suncong Zheng, Jiaming Xu +4 more
A neural network based framework for relation classification that can simultaneously learn the relation pattern's information and the semantic properties of given entities effectively is proposed.
Proceedings of the Thirteenth Conference on Computational Natural Language Learning
44 Citations2009Suzanne Stevenson, Xavier Carreras
Lecture notes in computer scienceMultiple Classifier Systems
39 Citations2004MARCIALIS, GIAN LUCA, Fabio Roli
A Feature-Enriched Tree Kernel for Relation Extraction
30 Citations2014Le Sun, Xianpei Han
A new tree kernel, called feature-enriched tree kernel (FTK), is proposed, which can enhance the traditional tree kernel by refining the syntactic tree representation by annotating each tree node with a set of discriminant features.
arXiv (Cornell University)Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling
24 Citations2015Kun Xu, Yansong Feng +2 more
Proceedings of the 2013 workshop on Automated knowledge base construction
16 Citations2013
The workshop on automated knowledge base construction (AKBC) is the venue for sharing the latest research in the area of knowledge extraction, and focuses exclusively on short, visionary papers, even if the experimentation is still rudimentary.
NeurocomputingA distributed meta-learning system for Chinese entity relation extraction
14 Citations2014Lishuang Li, Jing Zhang +3 more
A distributed meta- learning method which incorporates the distributed system and the meta-learning strategy for Chinese entity relation extraction is presented, which is effective and better than some other work.