Compositional Learning of Embeddings for Relation Paths in Knowledge Base and Text
Published 1 January 2016Open access
Kristina Toutanova, Victoria Lin, Wen-tau Yih, Hoifung Poon, Chris Quirk
Citations141
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper proposes the first exact dynamic programming algorithm which enables efficient incorporation of all relation paths of bounded length, while modeling both relation types and intermediate nodes in the compositional path representations.
Abstract
Modeling relation paths has offered significant gains in embedding models for knowledge base (KB) completion.However, enumerating paths between two entities is very expensive, and existing approaches typically resort to approximation with a sampled subset.This problem is particularly acute when text is jointly modeled with KB relations and used to provide direct evidence for facts mentioned in it.In this paper, we propose the first exact dynamic programming algorithm which enables efficient incorporation of all relation paths of bounded length, while modeling both relation types and intermediate nodes in the compositional path representations.We conduct a theoretical analysis of the efficiency gain from the approach.Experiments on two datasets show that it addresses representational limitations in prior approaches and improves accuracy in KB completion.
Keywords
Computer Science
Translating embeddings for modeling multi-relational data
5,178 Citations2015Antoine Bordes, Nicolas Usunier +2 more
TransE is proposed, a method which models relationships by interpreting them as translations operating on the low-dimensional embeddings of the entities, which proves to be powerful since extensive experiments show that TransE significantly outperforms state-of-the-art methods in link prediction on two knowledge bases.
Freebase
4,892 Citations2008Kurt Bollacker, Colin Evans +3 more
MQL provides an easy-to-use object-oriented interface to the tuple data in Freebase and is designed to facilitate the creation of collaborative, Web-based data-oriented applications.
Lecture notes in computer scienceDBpedia: A Nucleus for a Web of Open Data
4,675 Citations2007Sören Auer, Christian Bizer +4 more
The extraction of the DBpedia datasets is described, and how the resulting information is published on the Web for human-andmachine-consumption and how DBpedia could serve as a nucleus for an emerging Web of open data.
IEEE International Conference on Neural NetworksA direct adaptive method for faster backpropagation learning: the RPROP algorithm
3,927 Citations2002Martin Riedmiller, Heinrich Braun
A learning algorithm for multilayer feedforward networks, RPROP (resilient propagation), is proposed that performs a local adaptation of the weight-updates according to the behavior of the error function to overcome the inherent disadvantages of pure gradient-descent.
Distant supervision for relation extraction without labeled data
2,913 Citations2009Mike D. Mintz, Steven Bills +2 more
This work investigates an alternative paradigm that does not require labeled corpora, avoiding the domain dependence of ACE-style algorithms, and allowing the use of corpora of any size.
arXiv (Cornell University)Embedding Entities and Relations for Learning and Inference in Knowledge Bases
2,051 Citations2014d'Amato, Claudia, Wen-tau Yih +2 more
It is found that embeddings learned from the bilinear objective are particularly good at capturing relational semantics and that the composition of relations is characterized by matrix multiplication.
Reasoning With Neural Tensor Networks for Knowledge Base Completion
1,669 Citations2013Richard Socher, Danqi Chen +2 more
An expressive neural tensor network suitable for reasoning over relationships between two entities given a subset of the knowledge base is introduced and performance can be improved when entities are represented as an average of their constituting word vectors.
Nucleic Acids ResearchPID: the Pathway Interaction Database
1,654 Citations2008Carl F. Schaefer, Kira Anthony +5 more
A Three-Way Model for Collective Learning on Multi-Relational Data
1,647 Citations2011Maximilian Nickel, Volker Tresp +1 more
This work presents a novel approach to relational learning based on the factorization of a three-way tensor that is able to perform collective learning via the latent components of the model and provide an efficient algorithm to compute the factorizations.
Semantic Parsing on Freebase from Question-Answer Pairs
1,591 Citations2013Jonathan Berant, Andrew Chou +2 more
This paper trains a semantic parser that scales up to Freebase and outperforms their state-of-the-art parser on the dataset of Cai and Yates (2013), despite not having annotated logical forms.
Observed versus latent features for knowledge base and text inference
1,043 Citations2015Kristina Toutanova, Danqi Chen
It is shown that the observed features model is most effective at capturing the information present for entity pairs with textual relations, and a combination of the two combines the strengths of both model types.
Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base
796 Citations2015Wen-tau Yih, Ming‐Wei Chang +2 more
This work proposes a novel semantic parsing framework for question answering using a knowledge base that leverages the knowledge base in an early stage to prune the search space and thus simplifies the semantic matching problem.
Representing Text for Joint Embedding of Text and Knowledge Bases
761 Citations2015Kristina Toutanova, Danqi Chen +4 more
A model is proposed that captures the compositional structure of textual relations, and jointly optimizes entity, knowledge base, and textual relation representations, and significantly improves performance over a model that does not share parameters among textual relations with common sub-structure.
Multi-instance Multi-label Learning for Relation Extraction
682 Citations2012Mihai Surdeanu, Julie Tibshirani +2 more
This work proposes a novel approach to multi-instance multi-label learning for RE, which jointly models all the instances of a pair of entities in text and all their labels using a graphical model with latent variables that performs competitively on two difficult domains.
Machine LearningRelational retrieval using a combination of path-constrained random walks
630 Citations2010Ni Lao, William W. Cohen
A novel learnable proximity measure is described which instead uses one weight per edge label sequence: proximity is defined by a weighted combination of simple “path experts”, each corresponding to following a particular sequence of labeled edges.
Modeling Relation Paths for Representation Learning of Knowledge Bases
588 Citations2015Yankai Lin, Zhiyuan Liu +4 more
This model considers relation paths as translations between entities for representation learning, and addresses two key challenges: (1) Since not all relation paths are reliable, it design a path-constraint resource allocation algorithm to measure the reliability of relation paths and (2) represents relation paths via semantic composition of relation embeddings.
CORE Scholar (Wright State University)Relation Extraction with Matrix Factorization and Universal Schemas
587 Citations2013Sebastian Riedel, Limin Yao +2 more
Traversing Knowledge Graphs in Vector Space
274 Citations2015Kelvin Guu, J. J. Miller +1 more
It is demonstrated that compositional training acts as a novel form of structural regularization, reliably improving performance across all base models (reducing errors by up to 43%) and achieving new state-of-the-art results.
Compositional Vector Space Models for Knowledge Base Completion
235 Citations2015Arvind Neelakantan, Benjamin Roth +1 more
This paper presents an approach that reasons about conjunctions of multi-hop relations non-atomically, composing the implications of a path using a recurrent neural network (RNN) that takes as inputs vector embeddings of the binary relation in the path.
Typed Tensor Decomposition of Knowledge Bases for Relation Extraction
191 Citations2014Kai-Wei Chang, Wen-tau Yih +2 more
A tensor decomposition approach for knowledge base embedding that is highly scalable, and is especially suitable for relation extraction by leveraging relational domain knowledge about entity type information, which is significantly faster than previous approaches and better able to discover new relations missing from the database.
Incorporating Vector Space Similarity in Random Walk Inference over Knowledge Bases
179 Citations2014Matt Gardner, Partha Talukdar +2 more
A new technique for combining KB relations and surface text into a single graph representation that is much more compact than graphs used in prior work is presented, and how to incorporate vector space similarity into random walk inference over KBs is described.
Open Domain Question Answering via Semantic Enrichment
106 Citations2015Huan Sun, Hao Ma +4 more
A new QA system that mines answers directly from the Web, and meanwhile employs KBs as a significant auxiliary to further boost the QA performance, and makes the first attempt to link answer candidates to entities in Freebase, during answer candidate generation.
Composing Relationships with Translations
96 Citations2015Alberto García-Durán, Antoine Bordes +1 more
An extension of TransE is proposed that learns to explicitly model composition of relationships via the addition of their corresponding translation vectors and it is shown empirically that this allows to improve performance for predicting single relationships as well as compositions of pairs of them.
Improving Learning and Inference in a Large Knowledge-Base using Latent Syntactic Cues
95 Citations2013Matt Gardner, Partha Talukdar +2 more
For the first time, it is demonstrated that addition of edges labeled with latent features mined from a large dependency parsed corpus of 500 million Web documents can significantly outperform previous PRAbased approaches on the KB inference task.
BioinformaticsLiterome: PubMed-scale genomic knowledge base in the cloud
87 Citations2014Hoifung Poon, Chris Quirk +2 more
The Literome project has developed an automatic curation system to extract genomic knowledge from PubMed articles and made this knowledge available in the cloud with a Web site to facilitate browsing, searching and reasoning.
FigshareReading The Web with Learned Syntactic-Semantic Inference Rules
83 Citations2018Ni Lao, Amarnag Subramanya +2 more
A distributed, Web-scale implementation of a path-constrained random walk model that learns syntactic-semantic inference rules for binary relations from a graph representation of the parsed text and the knowledge base is described.
Nature PrecedingsPID: The Pathway Interaction Database
43 Citations2008Carl F. Schaefer, Kira Anthony +5 more
DISTANT SUPERVISION FOR CANCER PATHWAY EXTRACTION FROM TEXT
41 Citations2014Hoifung Poon, Kristina Toutanova +1 more
This is the first attempt to formulate the distant supervision problem for pathway extraction and apply a state-of-the-art method to extracting pathway interactions from PubMed abstracts and shows that distant supervision can effectively compensate for the lack of annotation.
The Reading TeacherReading the Web
9 Citations2015David M. Salyer
The significance of Internet Guided Reading is that it provides one successful instructional strategy for the primary classroom teacher that supports young children at various levels of proficiency with print and the Internet in learning to read informational texts on the Web.