A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction
Published 1 January 2019Open access
Mengjie Zhao, Hinrich Schütze
Citations11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new method for sentiment lexicon induction that is designed to be applicable to the entire range of typological diversity of the world’s languages is presented, to use Byte Pair Encodings as basic units for multilingual embeddings.
Abstract
We present a new method for sentiment lex- icon induction that is designed to be appli- cable to the entire range of typological di- versity of the world’s languages. We eval- uate our method on Parallel Bible Corpus+ (PBC+), a parallel corpus of 1593 languages. The key idea is to use Byte Pair Encodings (BPEs) as basic units for multilingual em- beddings. Through zero-shot transfer from English sentiment, we learn a seed lexicon for each language in the domain of PBC+. Through domain adaptation, we then gener- alize the domain-specific lexicon to a general one. We show – across typologically diverse languages in PBC+ – good quality of seed and general-domain sentiment lexicons by intrin- sic and extrinsic and by automatic and human evaluation. We make freely available our code, seed sentiment lexicons for all 1593 languages and induced general-domain sentiment lexi- cons for 200 languages
Keywords
Computer Science
Glove: Global Vectors for Word Representation
33,769 Citations2014Jeffrey Pennington, Richard Socher +1 more
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
arXiv (Cornell University)Distributed Representations of Words and Phrases and their Compositionality
18,086 Citations2013Tomáš Mikolov, Ilya Sutskever +3 more
This paper presents a simple method for finding phrases in text, and shows that learning good vector representations for millions of phrases is possible and describes a simple alternative to the hierarchical softmax called negative sampling.
Transactions of the Association for Computational LinguisticsEnriching Word Vectors with Subword Information
9,784 Citations2017Piotr Bojanowski, Édouard Grave +2 more
A new approach based on the skipgram model, where each word is represented as a bag of character n-grams, with words being represented as the sum of these representations, which achieves state-of-the-art performance on word similarity and analogy tasks.
Mining and summarizing customer reviews
7,714 Citations2004Minqing Hu, Bing Liu
This research aims to mine and to summarize all the customer reviews of a product, and proposes several novel techniques to perform these tasks.
now publishers, Inc. eBooksOpinion Mining and Sentiment Analysis
6,745 Citations2008Bo Pang, Lillian Lee
Proceedings of the International AAAI Conference on Web and Social MediaVADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text
5,749 Citations2014Cecelia Hutto, Éric Gilbert
Interestingly, using the authors' parsimonious rule-based model to assess the sentiment of tweets, it is found that VADER outperforms individual human raters, and generalizes more favorably across contexts than any of their benchmarks.
arXiv (Cornell University)Distributed Representations of Sentences and Documents
5,128 Citations2014Quoc V. Le, Tomáš Mikolov
Paragraph Vector is an unsupervised algorithm that learns fixed-length feature representations from variable-length pieces of texts, such as sentences, paragraphs, and documents, and its construction gives the algorithm the potential to overcome the weaknesses of bag-of-words models.
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
4,863 Citations1999John Platt
The output of a lassi(cid:12)er should be a alibrated posterior probability to enable post-pro essing and a method to train a kernel lassi with a logit link and a regularized maximum likelihood is proposed.
Recognizing contextual polarity in phrase-level sentiment analysis
3,379 Citations2005Theresa Wilson, Janyce Wiebe +1 more
A new approach to phrase-level sentiment analysis is presented that first determines whether an expression is neutral or polar and then disambiguates the polarity of the polar expressions.
SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining.
2,652 Citations2010Stefano Baccianella, Andrea Esuli +1 more
This work discusses SENTIWORDNET 3.0, a lexical resource explicitly devised for supporting sentiment classification and opinion mining applications, and reports on the improvements concerning aspect (b) that it embodies with respect to version 1.0.
Computational IntelligenceCROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON
2,584 Citations2012Saif M. Mohammad, Peter D. Turney
It is shown how the combined strength and wisdom of the crowds can be used to generate a large, high‐quality, word–emotion and word–polarity association lexicon quickly and inexpensively.
arXiv (Cornell University)Cross-lingual Language Model Pretraining
1,616 Citations2019Guillaume Lample, Alexis Conneau
This work proposes two methods to learn cross-lingual language models (XLMs): one unsupervised that only relies on monolingual data, and one supervised that leverages parallel data with a new cross-lingsual language model objective.
Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification
1,167 Citations2014Duyu Tang, Furu Wei +4 more
Three neural networks are developed to effectively incorporate the supervision from sentiment polarity of text (e.g. sentences or tweets) in their loss functions and the performance of SSWE is improved by concatenating SSWE with existing feature set.
arXiv (Cornell University)Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
770 Citations2018Mikel Artetxe, Holger Schwenk
An architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts using a single BiLSTM encoder with a shared byte-pair encoding vocabulary for all languages, coupled with an auxiliary decoder and trained on publicly available parallel corpora.
Learning bilingual word embeddings with (almost) no bilingual data
488 Citations2017Mikel Artetxe, Gorka Labaka +1 more
This work further reduces the need of bilingual resources using a very simple self-learning approach that can be combined with any dictionary-based mapping technique, and works with as little bilingual evidence as a 25 word dictionary or even an automatically generated list of numerals.
SemEval-2015 Task 10: Sentiment Analysis in Twitter
366 Citations2015Sara Rosenthal, Preslav Nakov +4 more
The 2015 iteration of the SemEval shared task on Sentiment Analysis in Twitter was the most popular sentiment analysis shared task to date with more than 40 teams participating in each of the last three years.
Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora
359 Citations2016William L. Hamilton, Kevin B. Clark +2 more
The approach achieves state-of-the-art performance on inducing sentiment lexicons from domain-specific corpora and that the purely corpus-based approach outperforms methods that rely on hand-curated resources (e.g., WordNet).
Proceedings of the AAAI Conference on Artificial IntelligenceGeneralizing and Improving Bilingual Word Embedding Mappings with a Multi-Step Framework of Linear Transformations
233 Citations2018Mikel Artetxe, Gorka Labaka +1 more
A multi-step framework of linear transformations that generalizes a substantial body of previous work is proposed that allows new insights into the behavior of existing methods, including the effectiveness of inverse regression, and design a novel variant that obtains the best published results in zero-shot bilingual lexicon extraction.
Multilingual Language Processing From Bytes
220 Citations2016Dan Gillick, Cliff Brunk +2 more
An LSTM-based model that reads text as bytes and outputs span annotations of the form [start, length, label] where start positions, lengths, and labels are separate entries in the authors' vocabulary is described.
PLoS ONEMultilingual Twitter Sentiment Classification: The Role of Human Annotators
194 Citations2016Igor Mozetič, Miha Grćar +1 more
It is shown that the model performance approaches the inter-annotator agreement when the size of the training set is sufficiently large and there is strong evidence that humans perceive the sentiment classes (negative, neutral, and positive) as ordered.
Lecture notes in computer scienceCollecting Evaluative Expressions for Opinion Extraction
189 Citations2005Nozomi Kobayashi, Kentaro Inui +3 more
A semi-automatic method that uses particular cooccurrence patterns of evaluated subjects, focused attributes and value expressions to accelerate the process of collecting evaluative expressions by applying a text mining technique is proposed.
A Large Scale Arabic Sentiment Lexicon for Arabic Opinion Mining
182 Citations2014Gilbert Badaro, Ramy Baly +3 more
This paper produces the first publicly available large scale Standard Arabic sentiment lexicon (ArSenL) using a combination of existing resources: ESWN, Arabic WordNet, and the Standard Arabic Morphological Analyzer (SAMA).
Charagram: Embedding Words and Sentences via Character n-grams
168 Citations2016John Wieting, Mohit Bansal +2 more
It is demonstrated that Charagram embeddings outperform more complex architectures based on character-level recurrent and convolutional neural networks, achieving new state-of-the-art performance on several similarity tasks.
Building Sentiment Lexicons for All Major Languages
158 Citations2014Yanqing Chen, Steven Skiena
This paper addresses the lexicon gap in a multilingual world by building high-quality sentiment lexicons for 136 major languages and integrates a variety of linguistic resources to produce an immense knowledge graph.
Learning Sentiment Lexicons in Spanish
147 Citations2012Verónica Pérez‐Rosas, Carmen Banea +1 more
It is shown that bridging the language gap using the multilingual sense-level aligned WordNet structure allows us to generate a high accuracy polarity lexicon comprising 1,347 entries, and a disjoint lower accuracy one encompassing 2,496 words.
arXiv (Cornell University)BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages
127 Citations2017Benjamin Heinzerling, Michael Strube
BPEmb, a collection of pre-trained subword unit embeddings in 275 languages, performs competitively, and for some languages bet- ter than alternative subword approaches, while requiring vastly fewer resources and no tokenization.
A survey of cross-lingual embedding models.
113 Citations2017Sebastian Ruder
This work surveys models that seek to learn cross-lingual embeddings and discusses them based on the type of approach and the nature of parallel data that they employ.
Information Processing & ManagementChinese word segmentation and its effect on information retrieval
100 Citations2002Schubert Foo, Hui Li
The findings reveal that the segmentation approach has an effect on IR effectiveness and better IR results are obtained by using the same method for query and document processing as this increase the probability of the query-document match.
Language Resources and EvaluationFEEL: a French Expanded Emotion Lexicon
88 Citations2016Amine Abdaoui, Jérôme Azé +2 more
The elaboration and the evaluation of a new French lexicon considering both polarity and emotion, based on the semi-automatic translation and expansion to synonyms of the English NRC Word Emotion Association Lexicon (NRC-EmoLex).
GermanPolarityClues: A Lexical Resource for German Sentiment Analysis
70 Citations2010Ulli Waltinger
Evaluating four different English and three different German sentiment resources in a comparative manner by combining a polarity-based feature selection with SVM-based machine learning classifier shows that the size of dictionaries clearly correlate to polarity, but this property does not correlate to classification accuracy.
A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments
66 Citations2017Omer Levy, Anders Søgaard +1 more
It is suggested that adding additional sources of information, which go beyond the traditional signal of bilingual sentence-aligned corpora, may substantially improve cross-lingual word embeddings, and that future baselines should at least take such features into account.
SLSA: A Sentiment Lexicon for Standard Arabic
58 Citations2015Ramy Eskander, Owen Rambow
A large-scale Standard Arabic Sentiment Lexicon (SLSA) that is publicly available for free and avoids the deficiencies in the current resources and has the highest up-to-date reported coverage.
INESC-ID: A Regression Model for Large Scale Twitter Sentiment Lexicon Induction
38 Citations2015Silvio Amir, Ramón Fernández Astudillo +4 more
This work presents the approach followed by INESCID in the SemEval 2015 Twitter Sentiment Analysis challenge, and achieves the top ranking submission, attesting the general adequacy of the proposed approach.
Computational LinguisticsCross-lingual Sentiment Lexicon Learning With Bilingual Word Graph Label Propagation
36 Citations2015Dehong Gao, Furu Wei +3 more
This article formalizes the task of cross-lingual sentiment lexicon learning as a learning problem on a bilingual word graph, and shows that both synonym and antonym word relations can be used to build the intra-language relation.
Improving Claim Stance Classification with Lexical Knowledge Expansion and Context Utilization
26 Citations2017Roy Bar-Haim, Lilach Edelstein +2 more
This work shows that both accuracy and coverage can be significantly improved through automatic expansion of the initial lexicon, and developed a set of contextual features that further improves the state-of-the-art for this task.
Nonsymbolic Text Representation
22 Citations2017Hinrich Schütze
This work introduces the first generic text representation model that is completely nonsymbolic, i.e., it does not require the availability of a segmentation or tokenization method that attempts to identify words or other symbolic units in text.
Embedding Learning Through Multilingual Concept Induction
20 Citations2018Philipp Dufter, Mengjie Zhao +3 more
An extensive experimental evaluation on crosslingual word similarity and sentiment analysis indicates that concept-based multilingual embedding learning performs better than previous approaches.
Waseda University Repository (Waseda University)Building Online Corpora of Philippine Languages
11 Citations2009Shirley N. Dita, Rachel Edita Roxas +1 more
This paper aims at describing the building of the online corpora on Philippine languages as part of theOnline repository system called Palito, which would consider text, speech and video forms, and the corresponding development of automated tools for language analysis of these various forms.