Unsupervised feature learning for sentiment classification of short documents
LDV-Forum/Journal for language technology and computational linguisticsPublished 1 July 2014Open access
Simone Albertini, Alessandro Zamberletti, Ignazio Gallo
Citations4
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
The rapid growth of Web information led to an increasing amount of user-generated content, such as customer reviews of products, forum posts and blogs.In this paper we face the task of assigning a sentiment polarity to user-generated short documents to determine whether each of them communicates a positive or negative judgment about a subject.The method we propose exploits a Growing Hierarchical Self-Organizing Map as feature learning algorithm to obtain a sparse encoding of the input data.The encoded documents are subsequently given as input to a Support Vector Machine classifier that assigns them a polarity label.Unlike other works on opinion mining, our model does not exploit a priori hypotheses involving special words, phrases or language constructs typical of certain domains.Using a dataset composed by customer reviews of products, our experimental results prove that the proposed method can overcome other state-of-the-art feature learning approaches.
Keywords
Computer Science
Machine LearningSupport-vector networks
40,463 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
NatureLearning representations by back-propagating errors
30,885 Citations1986David E. Rumelhart, Geoffrey E. Hinton +1 more
Back-propagation repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector, which helps to represent important features of the task domain.
ScienceReducing the Dimensionality of Data with Neural Networks
20,953 Citations2006Geoffrey E. Hinton, Ruslan Salakhutdinov
Neural ComputationA Fast Learning Algorithm for Deep Belief Nets
16,404 Citations2006Geoffrey E. Hinton, Simon Osindero +1 more
A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.
International Conference on Machine LearningRectified Linear Units Improve Restricted Boltzmann Machines
13,244 Citations2010Vinod Nair, Geoffrey E. Hinton
Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.
Mining and summarizing customer reviews
7,714 Citations2004Minqing Hu, Bing Liu
This research aims to mine and to summarize all the customer reviews of a product, and proposes several novel techniques to perform these tasks.
Extracting and composing robust features with denoising autoencoders
7,343 Citations2008Pascal Vincent, Hugo Larochelle +2 more
This work introduces and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern.
now publishers, Inc. eBooksOpinion Mining and Sentiment Analysis
6,745 Citations2008Bo Pang, Lillian Lee
Deep Sparse Rectifier Neural Networks
5,408 Citations2012Xavier Glorot, Antoine Bordes +1 more
This paper shows that rectifying neurons are an even better model of biological neurons and yield equal or better performance than hyperbolic tangent networks in spite of the hard non-linearity and non-dierentiabil ity.
Neural ComputationTraining Products of Experts by Minimizing Contrastive Divergence
5,017 Citations2002Geoffrey E. Hinton
A product of experts (PoE) is an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary because it is hard even to approximate the derivatives of the renormalization term in the combination rule.
Learning Word Vectors for Sentiment Analysis
3,299 Citations2011Andrew L. Maas, Raymond E. Daly +4 more
This work presents a model that uses a mix of unsupervised and supervised techniques to learn word vectors capturing semantic term--document information as well as rich sentiment content, and finds it out-performs several previously introduced methods for sentiment classification.
Lecture notes in computer scienceA Practical Guide to Training Restricted Boltzmann Machines
2,802 Citations2012Geoffrey E. Hinton
This guide is an attempt to share expertise at training restricted Boltzmann machines with other machine learning researchers.
An analysis of single-layer networks in unsupervised feature learning
2,457 Citations2011Adam Coates, Andrew Y. Ng +1 more
arXiv (Cornell University)Thumbs up? Sentiment Classification using Machine Learning Techniques
2,208 Citations2002Bo Pang, Lillian Lee +1 more
Extracting product features and opinions from reviews
1,720 Citations2005Ana-Maria Popescu, Oren Etzioni
Opine is introduced, an unsupervised information-extraction system which mines reviews in order to build a model of important product features, their evaluation by reviewers, and their relative quality across products.
Domain adaptation for large-scale sentiment classification: A deep learning approach
1,563 Citations2011Xavier Glorot, Antoine Bordes +1 more
A deep learning approach is proposed which learns to extract a meaningful representation for each review in an unsupervised fashion and clearly outperform state-of-the-art methods on a benchmark composed of reviews of 4 types of Amazon products.
Predicting the semantic orientation of adjectives
1,439 Citations1997Vasileios Hatzivassiloglou, Kathleen McKeown
A log-linear regression model uses constraints from conjunctions to predict whether conjoined adjectives are of same or different orientations, achieving 82% accuracy in this task when each conjunction is considered independently.
Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
1,198 Citations2011Richard Socher, Jeffrey Pennington +3 more
A novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions that outperform other state-of-the-art approaches on commonly used datasets, without using any pre-defined sentiment lexica or polarity shifting rules.
Movie review mining and summarization
909 Citations2006Zhuang Li, Jing Feng +1 more
A multi-knowledge based approach is proposed, which integrates WordNet, statistical analysis and movie knowledge, and the experimental results show the effectiveness of the proposed approach in movie review mining and summarization.
Extracting Product Features and Opinions from Reviews
490 Citations2007Ana-Maria Popescu, Orena Etzioni
IEEE Transactions on Neural NetworksThe growing hierarchical self-organizing map: exploratory analysis of high-dimensional data
487 Citations2002Andreas Rauber, Dieter Merkl +1 more
The motivation was to provide a model that adapts its architecture during its unsupervised training process according to the particular requirements of the input data, and by providing a global orientation of the independently growing maps in the individual layers of the hierarchy, navigation across branches is facilitated.
Just how mad are you? finding strong and weak opinion clauses
435 Citations2004Theresa Wilson, Janyce Wiebe +1 more
This paper presents the first experimental results classifying the strength of opinions and other types of subjectivity and classifies the subjectivity of deeply nested clauses using new syntactic features developed for opinion recognition.
Fully automatic lexicon expansion for domain-oriented sentiment analysis
411 Citations2006Hiroshi Kanayama, Tetsuya Nasukawa
This paper proposes an unsupervised lexicon building method for the detection of polar clauses, which convey positive or negative aspects in a specific domain, and its method is robust for corpora in diverse domains and for the size of the initial lexicon.
arXiv (Cornell University)Unsupervised Learning of Semantic Orientation from a Hundred-Billion-Word Corpus
361 Citations2002Peter D. Turney, Michael L. Littman
This paper introduces a simple algorithm for unsupervised learning of semantic orientation from extremely large corpora by issuing queries to a Web search engine and using pointwise mutual information to analyse the results.
Dependency Tree-based Sentiment Classification using CRFs with Hidden Variables
336 Citations2010Tetsuji Nakagawa, Kentaro Inui +1 more
Experimental results of sentiment classification of Japanese and English subjective sentences using conditional random fields with hidden variables showed that the method performs better than other methods based on bag-of-features.
Mining the Sentiment Expectation of Nouns Using Bootstrapping Method
39 Citations2011Miaomiao Wen, Yunfang Wu
An unsupervised bootstrapping method is proposed to generate a new type of affect knowledge base: the sentiment expectation of nouns, which could be automatically predicted with the output patterns.
International Joint Conference on Natural Language ProcessingPredicting Opinion Dependency Relations for Opinion Analysis
7 Citations2011Lun‐Wei Ku, Ting-Hao Huang +1 more
Annotation of gold answers of opinion syntactic structures on the parsing tree from Chinese Treebank and proposed methods to find their corresponding dependency relations on the dependency trees generated from the same sentence improve at least 8% of the performance of opinion analysis.