Lifelong Learning CRF for Supervised Aspect Extraction
Published 1 January 2017Open access
Lei Shu, Hu Xu, Bing Liu
Citations118
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper shows that if the system has performed aspect extraction from many past domains and retained their results as knowledge, Conditional Random Fields (CRF) can leverage this knowledge in a lifelong learning manner to extract in a new domain markedly better than the traditional CRF without using this prior knowledge.
Abstract
This paper makes a focused contribution to supervised aspect extraction. It shows that if the system has performed aspect extraction from many past domains and retained their results as knowledge, Conditional Random Fields (CRF) can leverage this knowledge in a lifelong learning manner to extract in a new domain markedly better than the traditional CRF without using this prior knowledge. The key innovation is that even after CRF training, the model can still improve its extraction with experiences in its applications.
Keywords
Computer Science
IEEE Transactions on Knowledge and Data EngineeringA Survey on Transfer Learning
23,164 Citations2009Sinno Jialin Pan, Qiang Yang
The relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift are discussed.
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Mining and summarizing customer reviews
7,714 Citations2004Minqing Hu, Bing Liu
This research aims to mine and to summarize all the customer reviews of a product, and proposes several novel techniques to perform these tasks.
Minds at UW (University of Wisconsin)Semi-Supervised Learning Literature Survey
3,868 Citations2005Xiaojin Zhu
The study clearly indicates that the common practice of stripwise precommercial thinning is unjustified, and the justification of heavy 'chessboard' thinning (with pruning) depends on whether the potential reduction in rotation length and the improvement in wood quality outweigh the discounted costs of pre-commercial thinning and selection and pruning of crop trees.
Encyclopedia of Machine Learning and Data MiningSentiment Analysis and Opinion Mining
3,319 Citations2017Lei Zhang, Bing Liu
Extracting product features and opinions from reviews
1,720 Citations2005Ana-Maria Popescu, Oren Etzioni
Opine is introduced, an unsupervised information-extraction system which mines reviews in order to build a model of important product features, their evaluation by reviewers, and their relative quality across products.
Computational LinguisticsOpinion Word Expansion and Target Extraction through Double Propagation
1,100 Citations2011Guang Qiu, Bing Liu +2 more
This article study two important problems, namely, opinion lexicon expansion and opinion target extraction, and proposes a method based on bootstrapping that outperforms these existing methods significantly.
Joint sentiment/topic model for sentiment analysis
972 Citations2009Chenghua Lin, Yulan He
A novel probabilistic modeling framework based on Latent Dirichlet Allocation (LDA), called joint sentiment/topic model (JST), which detects sentiment and topic simultaneously from text is proposed, which is fully unsupervised.
Movie review mining and summarization
909 Citations2006Zhuang Li, Jing Feng +1 more
A multi-knowledge based approach is proposed, which integrates WordNet, statistical analysis and movie knowledge, and the experimental results show the effectiveness of the proposed approach in movie review mining and summarization.
Topic sentiment mixture
813 Citations2007Qiaozhu Mei, Xu Ling +3 more
The proposed Topic-Sentiment Mixture (TSM) model can reveal the latent topical facets in a Weblog collection, the subtopics in the results of an ad hoc query, and their associated sentiments and could also provide general sentiment models that are applicable to any ad hoc topics.
Aspect and sentiment unification model for online review analysis
772 Citations2011Yohan Jo, Alice Oh
This paper proposes Sentence-LDA and extends it to Aspect and Sentiment Unification Model (ASUM), which incorporates aspect and sentiment together to model sentiments toward different aspects and shows that ASUM outperforms other generative models and comes close to supervised classification methods.
Latent aspect rating analysis on review text data
659 Citations2010Hongning Wang, Yue Lu +1 more
Empirical experiments show that the proposed latent rating regression model can effectively solve the problem of LARA, and that the detailed analysis of opinions at the level of topical aspects enabled by the proposed model can support a wide range of application tasks, such as aspect opinion summarization, entity ranking based on aspect ratings, and analysis of reviewers rating behavior.
A Joint Model of Text and Aspect Ratings for Sentiment Summarization
638 Citations2008Ivan Titov, Ryan McDonald
A statistical model is proposed which is able to discover corresponding topics in text and extract textual evidence from reviews supporting each of these aspect ratings, a fundamental problem in aspect-based sentiment summarization.
Encyclopedia of Machine Learning and Data MiningSentiment Analysis and Opinion Mining
622 Citations2016Lei Zhang, Bing Liu
An Unsupervised Aspect-Sentiment Model for Online Reviews
507 Citations2010Samuel Brody, Noémie Elhadad
An unsuper-vised system for extracting aspects and determining sentiment in review text is presented, which is simple and flexible with regard to domain and language, and takes into account the influence of aspect on sentiment polarity.
Extracting Product Features and Opinions from Reviews
490 Citations2007Ana-Maria Popescu, Orena Etzioni
TUbilio (Technical University of Darmstadt)Extracting Opinion Targets in a Single and Cross-Domain Setting with Conditional Random Fields
480 Citations2010Niklas Jakob, Iryna Gurevych
This paper model the problem as an information extraction task, which is addressed based on Conditional Random Fields (CRF), and employs the supervised algorithm by Zhuang et al. (2006), which represents the state-of-the-art on the employed data.
Synthesis lectures on artificial intelligence and machine learningLifelong Machine Learning
452 Citations2018Zhiyuan Chen, Bing Liu
The purpose of this second edition is to expand the definition of lifelong learning, update the content of several chapters, and add a new chapter about continual learning in deep neural networks—which has been actively researched over the past two or three years.
Lifelong Learning Algorithms
422 Citations1998Sebastian Thrun
Machine learning has not yet succeeded in the design of robust learning algorithms that generalize well from very small datasets, while humans often generalize correctly from only a single training example, even if the number of potentially relevant features is large.
Singapore Management University Institutional Knowledge (InK) (Singapore Management University)Jointly Modeling Aspects and Opinions with a MaxEnt-LDA Hybrid
414 Citations2010Xin Zhao, Jing Jiang +2 more
This paper proposes a MaxEnt-LDA hybrid model to jointly discover both aspects and aspect-specific opinion words and shows that with a relatively small amount of training data, this model can effectively identify aspect and opinion words simultaneously.
Phrase dependency parsing for opinion mining
365 Citations2009Yuanbin Wu, Qi Zhang +2 more
A novel approach for mining opinions from product reviews is presented, where it converts opinion mining task to identify product features, expressions of opinions and relations between them by taking advantage of the observation that a lot of product features are phrases.
Aspect Extraction through Semi-Supervised Modeling
345 Citations2012Arjun Mukherjee, Bing Liu
This paper proposes two statistical models to solve the seeded problem of aspect extraction in a different setting where the user provides some seed words for a few aspect categories and the model extracts and clusters aspect terms into categories simultaneously.
Lifelong Machine Learning Systems: Beyond Learning Algorithms
288 Citations2013Daniel Silver, Qiang Yang +1 more
It is proposed that it is now appropriate for the AI community to move beyond learning algorithms to more seriously consider the nature of systems that are capable of learning over a lifetime.
A Rule-Based Approach to Aspect Extraction from Product Reviews
285 Citations2014Soujanya Poria, Erik Cambria +3 more
This work aims to solve the problem of aspect extraction from product reviews by proposing a novel rule-based approach that exploits common-sense knowledge and sentence dependency trees to detect both explicit and implicit aspects.
Extracting and Ranking Product Features in Opinion Documents
267 Citations2010Lei Zhang, Bing Liu +2 more
The problem is formulated as a bipartite graph and the well-known web page ranking algorithm HITS is used to find important features and rank them high and demonstrates promising results on diverse real-life datasets.
Proceedings of the AAAI Conference on Artificial IntelligenceSentiment Analysis with Global Topics and Local Dependency
249 Citations2010Fangtao Li, Minlie Huang +1 more
This paper proposes a major departure from the previous approaches by assuming that the sentiments are related to the topic in the document, and puts forward a joint sentiment and topic model, i.e. Sentiment-LDA, and shows that exploiting the sentiment dependency is clearly advantageous, and that the Dependency-Sentiment- LDA is an effective approach for sentiment analysis.
ELLA: An Efficient Lifelong Learning Algorithm
233 Citations2013Paul Ruvolo, Eric Eaton
The proposed Efficient Lifelong Learning Algorithm (ELLA) maintains a sparsely shared basis for all task models, transfers knowledge from the basis to learn each new task, and refines the basis over time to maximize performance across all tasks.
Aspect Extraction with Automated Prior Knowledge Learning
232 Citations2014Zhiyuan Chen, Arjun Mukherjee +1 more
It is shown that in the big data era, without any user input, it is possible to learn prior knowledge automatically from a large amount of review data available on the Web, which can then be used by a topic model to discover more coherent aspects.
Open Domain Targeted Sentiment
207 Citations2013Margaret Mitchell, Jacqui Aguilar +2 more
The intuition behind this work is that sentiment expressed towards an entity, targeted sentiment, may be viewed as a span of sentiment expressed across the entity, and this representation allows us to model sentiment detection as a sequence tagging problem, jointly discovering people and organizations along with whether there is sentiment directed towards them.
ILDA
202 Citations2011Samaneh Moghaddam, Martin Ester
This paper introduces Interdependent Latent Dirichlet Allocation (ILDA) model, a probabilistic graphical models which aim to extract aspects and corresponding ratings of products from online reviews and conducts experiments on a real life dataset, Epinions.com.
Lifelong Learning for Sentiment Classification
164 Citations2015Zhiyuan Chen, Nianzu Ma +1 more
The proposed LL approach adopts a Bayesian optimization framework based on stochastic gradient descent, which demonstrates that lifelong learning is a promising research direction.
International Conference on Machine LearningTopic Modeling using Topics from Many Domains, Lifelong Learning and Big Data
162 Citations2014Zhiyuan Chen, Bing Liu
A novel method to mine prior knowledge dynamically in the modeling process, and then a new topic model to use the knowledge to guide the model inference, which offers a novel lifelong learning algorithm for topic discovery.
Synthesis lectures on artificial intelligence and machine learningLifelong Machine Learning
154 Citations2017Zhiyuan Chen, Bing Liu
As statistical machine learning matures, it is time to make a major effort to break the isolated learning tradition and to study lifelong learning to bring machine learning to new heights.
Proceedings of the AAAI Conference on Artificial IntelligenceImproving Opinion Aspect Extraction Using Semantic Similarity and Aspect Associations
141 Citations2016Qian Liu, Bing Liu +3 more
A novel unsupervised approach to make a major improvement in aspect extraction is proposed based on the framework of lifelong learning and is implemented with two forms of recommendations that are based on semantic similarity and aspect associations respectively.
Hierarchical Sequential Learning for Extracting Opinions and Their Attributes
128 Citations2010Yejin Choi, Claire Cardie
This paper applies a hierarchical parameter sharing technique using Conditional Random Fields for fine-grained opinion analysis, jointly detecting the boundaries of opinion expressions as well as determining two of their key attributes --- polarity and intensity.
Multi-aspect opinion polling from textual reviews
122 Citations2009Jingbo Zhu, Huizhen Wang +2 more
An unsupervised approach to aspect-based opinion polling from raw textual reviews without explicit ratings is presented and a multi-aspect bootstrapping algorithm is proposed to learn from unlabeled data aspect-related terms of each aspect to be used for aspect identification.
Bootstrapping Both Product Features and Opinion Words from Chinese Customer Reviews with Cross-Inducing
93 Citations2008Bo Wang, Houfeng Wang
Empirical results on three kinds of product reviews indicate the effectiveness of the bootstrapping iterative learning strategy proposed, and a mapping function from opinion words to features is proposed to identify implicit features in sentence.
Opinion target extraction using partially-supervised word alignment model
85 Citations2013Kang Liu, Liheng Xu +2 more
A novel approach to extract opinion targets by using partially-supervised word alignment model (PSWAM), which can capture opinion relations more precisely through partial supervision from partial alignment links and outperforms state-of-the-art methods.
Institutional Research Information System (Università degli Studi di Trento)Syntactic and Semantic Structure for Opinion Expression Detection
65 Citations2010Richard Johansson, Alessandro Moschitti
It is demonstrated that relational features derived from dependency-syntactic and semantic role structures are useful for the task of detecting opinionated expressions in natural-language text, significantly improving over conventional models based on sequence labeling with local features.
Collective Opinion Target Extraction in Chinese Microblogs
61 Citations2013Xinjie Zhou, Xiaojun Wan +1 more
This paper proposes an unsupervised label propagation algorithm to address the problem of extracting opinion targets of Chinese microblog messages and shows the effectiveness of the framework and algorithms.
Lifelong-RL: Lifelong Relaxation Labeling for Separating Entities and Aspects in Opinion Targets
60 Citations2016Lei Shu, Bing Liu +2 more
This paper proposes a novel algorithm, called Lifelong-RL, to solve the problem of opinion mining based on lifelong machine learning and relaxation labeling, and extensive experiments show that the proposed algorithm Lifetime-RL outperforms baseline methods markedly.
Fine Granular Aspect Analysis using Latent Structural Models
47 Citations2012Lei Fang, Minlie Huang
The model is able to find informative sentences that are closely related to some respects in a review, which may be helpful for aspect-level sentiment analysis such as aspect-oriented summarization and preliminary experiments demonstrate that the model obtains promising performance.
IEEE Intelligent SystemsCreating a Fine-Grained Corpus for Chinese Sentiment Analysis
40 Citations2014Yanyan Zhao, Bing Qin +1 more
The proposed fine-grained annotation scheme not only introduces cross-sentence and global sentiment information but also includes new sentence-level elements (such as "implicit aspect") that can provide a more fine- grained platform for researchers to study algorithms for advanced applications.