Multi-Step Regression Learning for Compositional Distributional Semantics
Oxford University Research Archive (ORA) (University of Oxford)Published 29 January 2013Open access
Edward Grefenstette, Georgiana Dinu, Yao-Zhong Zhang, Mehrnoosh Sadrzadeh, Marco Baroni
Citations57
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is argued in the analysis that the nature of this learning method also renders it suitable for solving more subtle problems compositional distributional models might face, and is found to outperform existing leading methods.
Abstract
We present a model for compositional distributional semantics related to the framework of Coecke et al. (2010), and emulating formal semantics by representing functions as tensors and arguments as vectors. We introduce a new learning method for tensors, generalising the approach of Baroni and Zamparelli (2010). We evaluate it on two benchmark data sets, and find it to outperform existing leading methods. We argue in our analysis that the nature of this learning method also renders it suitable for solving more subtle problems compositional distributional models might face.
Keywords
Computer Science
Psychological ReviewA solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.
6,039 Citations1997Thomas K. Landauer, Susan Dumais
A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LSA), is presented and used to successfully simulate such learning and several other psycholinguistic phenomena.
TechnometricsGeneralized Cross-Validation as a Method for Choosing a Good Ridge Parameter
3,770 Citations1979Gene H. Golub, Michael T. Heath +1 more
The method of generalized cross-validation (GCV) for choosing a good value for λ from the data is studied, which can be used in subset selection and singular value truncation methods for regression, and even to choose from among mixtures of these methods.
Neural ComputationProjected Gradient Methods for Nonnegative Matrix Factorization
1,776 Citations2007Chih‐Jen Lin
This letter proposes two projected gradient methods for nonnegative matrix factorization, both of which exhibit strong optimization properties and discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach.
Semantic Compositionality through Recursive Matrix-Vector Spaces
1,297 Citations2012Richard Socher, Brody Huval +2 more
A recursive neural network model that learns compositional vector representations for phrases and sentences of arbitrary syntactic type and length and can learn the meaning of operators in propositional logic and natural language is introduced.
Cognitive ScienceComposition in Distributional Models of Semantics
981 Citations2010Jeff Mitchell, Mirella Lapata
This article proposes a framework for representing the meaning of word combinations in vector space in terms of additive and multiplicative functions, and introduces a wide range of composition models that are evaluated empirically on a phrase similarity task.
Explorations in Automatic Thesaurus Discovery
757 Citations1994Gregory Grefenstette
The aim of this monograph is to provide a catalog of words and phrases used in ThesaurusGeneration, as well as some examples of other writers' work, which have been used in similar contexts.
Riemannian Manifolds: An Introduction to Curvature
753 Citations1997John M. Lee
Edinburgh Research Explorer (University of Edinburgh)Vector-based Models of Semantic Composition
666 Citations2008Jeff Mitchell, Mirella Lapata
Under this framework, a wide range of composition models are introduced which are evaluated empirically on a sentence similarity task and demonstrate that the multiplicative models are superior to the additive alternatives when compared against human judgments.
Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space
485 Citations2010Marco Baroni, Roberto Zamparelli
This work proposes an approach to adjective-noun composition (AN) for corpus-based distributional semantics that represents nouns as vectors and adjectives as data-induced (linear) functions over nominal vectors, and shows that the model significantly outperforms the rivals on the task of reconstructing AN vectors not seen in training.
A structured vector space model for word meaning in context
362 Citations2008Katrin Erk, Sebastian Padó
A novel structured vector space model is presented that makes it possible to integrate syntax into the computation of word meaning in context and performs at and above the state of the art for modeling the contextual adequacy of paraphrases.
Behavior Research MethodsExtracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD
309 Citations2012John A. Bullinaria, Joseph P. Levy
This article investigates the use of three further factors—namely, the application of stop-lists, word stemming, and dimensionality reduction using singular value decomposition (SVD)—that have been used to provide improved performance elsewhere and introduces an additional semantic task and explores the advantages of using a much larger corpus.
A Regression Model of Adjective-Noun Compositionality in Distributional Semantics
130 Citations2010Emiliano Guevara
This study leads to the conclusion that regression-based models of compositionality generally out-perform additive and multiplicative approaches, and also show a number of advantages that make them very promising for future research.
Edinburgh Research Explorer (University of Edinburgh)Measuring Distributional Similarity in Context
120 Citations2010Georgiana Dinu, Mirella Lapata
A probabilistic framework for measuring similarity in context that is based on the intuition that word meaning is represented as a probability distribution over a set of latent senses and is modulated by context is proposed.
Text, speech and language technologyConcrete Sentence Spaces for Compositional Distributional Models of Meaning
72 Citations2013Edward Grefenstette, Mehrnoosh Sadrzadeh +3 more
This paper provides a concrete method for implementing a linear meaning map, by constructing a corpus-based vector space for the type of sentence, based on structured vector spaces whereby meaning vectors of all sentences, regardless of their grammatical structure, live in the same vector space.
International Joint Conference on Natural Language ProcessingWord Meaning in Context: A Simple and Effective Vector Model
71 Citations2011Stefan Thater, Hagen Fürstenau +1 more
A model that represents word meaning in context by vectors which are modified according to the words in the target’s syntactic context, which outperforms all previous models on a word sense disambiguation task.
Cognitive ScienceData‐driven approaches to information access
50 Citations2003Susan Dumais
This paper summarizes three lines of research that are motivated by the practical problem of helping users find information from external data sources, most notably computers, that believe that solutions to practical information access problems can shed light on human knowledge representation and reasoning.