Estimating Linear Models for Compositional Distributional Semantics
Cineca Institutional Research Information System (Tor Vergata University)Published 23 August 2010Open access
Fabio Massimo Zanzotto, Ioannis Korkontzelos, Francesca Fallucchi, Suresh Manandhar
Citations126
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper proposes a novel approach to estimate parameters for a class of compositional distributional models: the additive models, and demonstrates that this approach outperforms existing methods for determining a good model for compositional distributionsal semantics.
Abstract
In distributional semantics studies, there is a growing attention in compositionally determining the distributional meaning of word sequences. Yet, compositional distributional models depend on a large set of parameters that have not been explored. In this paper we propose a novel approach to estimate parameters for a class of compositional distributional models: the additive models. Our approach leverages on two main ideas. Firstly, a novel idea for extracting compositional distributional semantics examples. Secondly, an estimation method based on regression models for multiple dependent variables. Experiments demonstrate that our approach outperforms existing methods for determining a good model for compositional distributional semantics. 1
Keywords
Computer ScienceBiochemistry, Genetics and Molecular Biology
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
Mathematical Proceedings of the Cambridge Philosophical SocietyA generalized inverse for matrices
4,495 Citations1955Roger Penrose
Journal of the Society for Industrial and Applied Mathematics Series B Numerical AnalysisCalculating the Singular Values and Pseudo-Inverse of a Matrix
1,773 Citations1965Gene H. Golub, W. Kahan
The use of the pseudo-inverse $A^I = V\Sigma ^I U^* $ to solve least squares problems in a way which dampens spurious oscillation and cancellation is mentioned.
Language and Cognitive ProcessesContextual correlates of semantic similarity
1,584 Citations1991George A. Miller, Walter G. Charles
A maximum-entropy-inspired parser
1,496 Citations2000Eugene Charniak
A new parser for parsing down to Penn tree-bank style parse trees that achieves 90.1% average precision/recall for sentences of length 40 and less and 89.5% when trained and tested on the previously established sections of the Wall Street Journal treebank is presented.
Computational LinguisticsDependency-Based Construction of Semantic Space Models
701 Citations2007Sebastian Padó, Mirella Lapata
This article presents a novel framework for constructing semantic spaces that takes syntactic relations into account, and introduces a formalization for this class of models, which allows linguistic knowledge to guide the construction process.
Edinburgh Research Explorer (University of Edinburgh)Vector-based Models of Semantic Composition
666 Citations2008Jeff Mitchell, Mirella Lapata
Under this framework, a wide range of composition models are introduced which are evaluated empirically on a sentence similarity task and demonstrate that the multiplicative models are superior to the additive alternatives when compared against human judgments.
Psychological ReviewRepresenting word meaning and order information in a composite holographic lexicon.
638 Citations2007Michael N. Jones, D. J. K. Mewhort
A computational model that builds a holographic lexicon representing both word meaning and word order from unsupervised experience with natural language demonstrates that a broad range of psychological data can be accounted for directly from the structure of lexical representations learned in this way, without the need for complexity to be built into either the processing mechanisms or the representations.
DIRT @SBT@discovery of inference rules from text
548 Citations2001Dekang Lin, Patrick Pantel
This paper proposes an unsupervised method for discovering inference rules from text, based on an extended version of Harris' Distributional Hypothesis, which states that words that occurred in the same contexts tend to be similar.
A structured vector space model for word meaning in context
362 Citations2008Katrin Erk, Sebastian Padó
A novel structured vector space model is presented that makes it possible to integrate syntax into the computation of word meaning in context and performs at and above the state of the art for modeling the contextual adequacy of paraphrases.
Distributional Structure
317 Citations1970Zellig S. Harris
This discussion will discuss how each language can be described in terms of a distributional structure, i.e. in Terms of the occurrence of parts relative to other parts, and how this description is complete without intrusion of other features such as history or meaning.
Is Knowledge-Free Induction of Multiword Unit Dictionary Headwords a Solved Problem?
149 Citations2001Patrick Schone, Daniel Jurafsky
This work seeks a knowledge-free method for inducing multiword units from text corpora for use as machine-readable dictionary headwords and uses Latent Semantic Analysis to make modest gains in performance, but shows the significant challenges encountered.
Computational LinguisticsDisambiguating Nouns, Verbs, and Adjectives Using Automatically Acquired Selectional Preferences
141 Citations2003Diana McCarthy, John Carroll
Although the preferences perform well in comparison with other unsupervised WSD systems on the same corpus, the results show that for many applications, further knowledge sources would be required to achieve an adequate level of accuracy and coverage.
A Simple, Similarity-based Model for Selectional Preferences
138 Citations2007Katrin Erk
A new, simple model for the automatic induction of selectional preferences, using corpus-based semantic similarity metrics, focuses on the task of semantic role labeling and shows lower error rates than both Resnik's WordNet-based model and the EM-based clustering model.
The Acquisition of Word Meaning through Global Lexical Co-occurrences
54 Citations2000Ping Li, Curt Burgess +1 more
Verb-particle constructions and lexical resources
41 Citations2003Aline Villavicencio
The use of Levin's ( 1993) classes of verbs are discussed as a means to obtain productive verb-particle constructions, and the issues involved in adopting such an approach are discussed.
Detecting compositionality in multi-word expressions
26 Citations2009Ioannis Korkontzelos, Suresh Manandhar
An unsupervised system is proposed to explore the hypothesis that sense induction can partition the context of MWEs into semantic uses and therefore aid in deciding compositionality, and evaluate the contribution of sense induction.
Standardised Evaluation of English Noun Compound Interpretation
11 Citations2008S Kim, Timothy Baldwin
SVD feature selection for probabilistic taxonomy learning
6 Citations2009Fallucchi Francesca, Fabio Massimo Zanzotto
A novel way to include unsupervised feature selection methods in probabilistic taxonomy learning models is proposed by leveraging on the computation of logistic regression to exploit unsuper supervised feature selection of singular value decomposition.