Concrete Sentence Spaces for Compositional Distributional Models of Meaning

This paper provides a concrete method for implementing a linear meaning map, by constructing a corpus-based vector space for the type of sentence, based on structured vector spaces whereby meaning vectors of all sentences, regardless of their grammatical structure, live in the same vector space.

Abstract

Coecke et al. (2010) developed a compositional model of meaning for distributional semantics, in which each word in a sentence has a meaning vector and the distributional meaning of the sentence is a function of the tensor products of the word vectors. Abstractly speaking, this function is the morphism corresponding to the grammatical structure of the sentence in the category of finite dimensional vector spaces. In this chapter, we provide a concrete method for implementing this linear meaning map by presenting an algorithm for computing representations for various syntactic classes which have functional types; this algorithm results in assigning concrete corpus-based vector spaces to the abstract type of 'sentence'. Our construction method is based on structured vector spaces whose basis vectors are pairs of words and grammatical roles. The concrete sentence spaces only depend on the types of the verbs of sentences; we use an embedding of these spaces and compare meanings of sentences with different grammatical structures by simply taking the inner product of their vectors in the bigger space. Our constructions are exemplified on a toy corpus.