Exploiting BERT for End-to-End Aspect-based Sentiment Analysis
Published 1 January 2019Open access
Xin Li, Lidong Bing, Wenxuan Zhang, Wai Lam
Citations320
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work builds a series of simple yet insightful neural baselines to deal with E2E-ABSA, and shows that even with a simple linear classification layer, the BERT-based architecture can outperform state-of-the-art works.
Abstract
In this paper, we investigate the modeling power of contextualized embeddings from pretrained language models, e.g. BERT, on the E2E-ABSA task. Specifically, we build a series of simple yet insightful neural baselines to deal with E2E-ABSA. The experimental results show that even with a simple linear classification layer, our BERT-based architecture can outperform state-of-the-art works. Besides, we also standardize the comparative study by consistently utilizing a hold-out development dataset for model selection, which is largely ignored by previous works. Therefore, our work can serve as a BERT-based benchmark for E2E-ABSA. 1
Keywords
Computer Science
Deep Residual Learning for Image Recognition
222,082 Citations2016Kaiming He, Xiangyu Zhang +2 more
This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
Neural ComputationLong Short-Term Memory
98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)MizAR 60 for Mizar 50
76,311 Citations2023Jakubův, Jan, Chvalovský, Karel +7 more
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)AI-Assisted Pipeline for Dynamic Generation of Trustworthy Health Supplement Content at Scale
45,681 Citations2018Kefallinos, Dionysios, Alexandris, Georgios +6 more
Glove: Global Vectors for Word Representation
33,769 Citations2014Jeffrey Pennington, Richard Socher +1 more
A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
32,525 Citations2019Jacob Devlin, Ming‐Wei Chang +2 more
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
24,447 Citations2014Kyunghyun Cho, Bart van Merriënboer +5 more
Qualitatively, the proposed RNN Encoder‐Decoder model learns a semantically and syntactically meaningful representation of linguistic phrases.
arXiv (Cornell University)Distributed Representations of Words and Phrases and their Compositionality
18,086 Citations2013Tomáš Mikolov, Ilya Sutskever +3 more
This paper presents a simple method for finding phrases in text, and shows that learning good vector representations for millions of phrases is possible and describes a simple alternative to the hierarchical softmax called negative sampling.
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Cognitive ScienceFinding Structure in Time
10,841 Citations1990Jeffrey L. Elman
A proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory and suggests a method for representing lexical categories and the type/token distinction is developed.
Neural Architectures for Named Entity Recognition
4,428 Citations2016Guillaume Lample, Miguel Ballesteros +3 more
Comunicacio presentada a la 2016 Conference of the North American Chapter of the Association for Computational Linguistics, celebrada a San Diego (CA, EUA) els dies 12 a 17 of juny 2016.
Encyclopedia of Machine Learning and Data MiningSentiment Analysis and Opinion Mining
3,319 Citations2017Lei Zhang, Bing Liu
Leibniz-Zentrum für Informatik (Schloss Dagstuhl)NER in Archival Finding Aids
3,270 Citations2021Zhiheng Huang, Wei Xu +1 more
This work has created a set of datasets to train Machine Learning algorithms to find named entities and geographical locations and tested them in several datasets and registered their precision and accuracy.
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
2,589 Citations2016Xuezhe Ma, Eduard Hovy
A novel neutral network architecture is introduced that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF, thus making it applicable to a wide range of sequence labeling tasks.
Attention-based LSTM for Aspect-level Sentiment Classification
2,344 Citations2016Yequan Wang, Minlie Huang +2 more
This paper reveals that the sentiment polarity of a sentence is not only determined by the content but is also highly related to the concerned aspect, and proposes an Attention-based Long Short-Term Memory Network for aspect-level sentiment classification.
SemEval-2014 Task 4: Aspect Based Sentiment Analysis
1,881 Citations2014Maria Pontiki, Dimitrios Galanis +4 more
SemEval2014 Task 4 aimed to foster research in the field of aspect-based sentiment analysis, where the goal is to identify the aspects of given target entities and the sentiment expressed for each aspect.
arXiv (Cornell University)Cross-lingual Language Model Pretraining
1,616 Citations2019Guillaume Lample, Alexis Conneau
This work proposes two methods to learn cross-lingual language models (XLMs): one unsupervised that only relies on monolingual data, and one supervised that leverages parallel data with a new cross-lingsual language model objective.
arXiv (Cornell University)A Structured Self-attentive Sentence Embedding
1,458 Citations2017Zhouhan Lin, Minwei Feng +5 more
An Empirical Exploration of Recurrent Network Architectures
1,402 Citations2015Rafał Józefowicz, Wojciech Zaremba +1 more
It is found that adding a bias of 1 to the LSTM's forget gate closes the gap between the L STM and the recently-introduced Gated Recurrent Unit (GRU) on some but not all tasks.
Zenodo (CERN European Organization for Nuclear Research)Neuromodulatory Control Networks (NCNs): A Biologically Inspired Architecture for Dynamic LLM Processing
1,395 Citations2025Morgan, Michael Christian
SemEval-2015 Task 12: Aspect Based Sentiment Analysis
1,152 Citations2015Maria Pontiki, Dimitrios Galanis +3 more
The task provided manually annotated reviews in three domains (restaurants, laptops and hotels), and a common evaluation procedure, to foster research beyond sentenceor text-level sentiment classification towards Aspect Based Sentiment Analysis.
Long Short-Term Memory-Networks for Machine Reading
1,071 Citations2016Jianpeng Cheng, Li Dong +1 more
A machine reading simulator which processes text incrementally from left to right and performs shallow reasoning with memory and attention and extends the Long Short-Term Memory architecture with a memory network in place of a single memory cell, offering a way to weakly induce relations among tokens.
Aspect Level Sentiment Classification with Deep Memory Network
1,035 Citations2016Duyu Tang, Bing Qin +1 more
A deep memory network for aspect level sentiment classification that explicitly captures the importance of each context word when inferring the sentiment polarity of an aspect, and is also fast.
Recurrent Attention Network on Memory for Aspect Sentiment Analysis
1,024 Citations2017Peng Chen, Zhongqian Sun +2 more
A novel framework based on neural networks to identify the sentiment of opinion targets in a comment/review that adopts multiple-attention mechanism to capture sentiment features separated by a long distance, so that it is more robust against irrelevant information.
Interactive Attention Networks for Aspect-Level Sentiment Classification
1,014 Citations2017Dehong Ma, Sujian Li +2 more
It is argued that both targets and contexts deserve special treatment and need to be learned their own representations via interactive learning and proposed the interactive attention networks (IAN) to interactively learn attentions in the contexts and targets.
arXiv (Cornell University)Unified language model pre-training for natural language understanding and generation
949 Citations2024Li Dong
A new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks that compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks.
Proceedings of the AAAI Conference on Artificial IntelligenceDiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
747 Citations2018Tao Shen, Tianyi Zhou +4 more
A novel attention mechanism in which the attention between elements from input sequence(s) is directional and multi-dimensional (i.e., feature-wise) and a light-weight neural net is proposed, based solely on the proposed attention without any RNN/CNN structure, which outperforms complicated RNN models on both prediction quality and time efficiency.
Encyclopedia of Machine Learning and Data MiningSentiment Analysis and Opinion Mining
622 Citations2016Lei Zhang, Bing Liu
Aspect Based Sentiment Analysis with Gated Convolutional Networks
621 Citations2018Wei Xue, Tao Li
A model based on convolutional neural networks and gating mechanisms, which is more accurate and efficient, and the novel Gated Tanh-ReLU Units can selectively output the sentiment features according to the given aspect or entity.
Proceedings of the AAAI Conference on Artificial IntelligenceTargeted Aspect-Based Sentiment Analysis via Embedding Commonsense Knowledge into an Attentive LSTM
608 Citations2018Yukun Ma, Haiyun Peng +1 more
A novel solution to targeted aspect-based sentiment analysis, which tackles the challenges of both aspect- based sentiment analysis and targeted sentiment analysis by exploiting commonsense knowledge by augmenting the LSTM network with a hierarchical attention mechanism.
arXiv (Cornell University)Unified Language Model Pre-training for Natural Language Understanding\n and Generation
539 Citations2019Li Dong, Nan Yang +7 more
Multi-grained Attention Network for Aspect-Level Sentiment Classification
513 Citations2018Feifan Fan, Yansong Feng +1 more
Experimental results show that the multi-grained attention network consistently outperforms the state-of-the-art methods on all three datasets, and the effectiveness of aspect alignment loss indicates the aspect-level interactions can bring extra useful information and further improve the performance.
Transformation Networks for Target-Oriented Sentiment Classification
500 Citations2018Xin Li, Lidong Bing +2 more
A new model is proposed to overcome the drawbacks of attention mechanism and the obstacles that block CNN to perform well in this sentiment classification task, and employs a CNN layer to extract salient features from the transformed word representations originated from a bi-directional RNN layer.
International Conference on Learning RepresentationsA Structured Self-Attentive Sentence Embedding.
364 Citations2017Zhouhan Lin, Minwei Feng +5 more
arXiv (Cornell University)Contributions to Legal Document Summarization: Judgments from the Portuguese Supreme Court of Justice
360 Citations2019Liu, Yang
arXiv (Cornell University)BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
358 Citations2019Hu Xu, Bing Liu +2 more
arXiv (Cornell University)Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence
350 Citations2019Chi Sun, Luyao Huang +1 more
307 Citations2019Hu Xu, Bing Liu +2 more
A novel post-training approach on the popular language model BERT to enhance the performance of fine-tuning of BERT for RRC and is applied to some other review-based tasks such as aspect extraction and aspect sentiment classification in aspect-based sentiment analysis.
Proceedings of the AAAI Conference on Artificial IntelligenceA Unified Model for Opinion Target Extraction and Target Sentiment Prediction
301 Citations2019Xin Li, Lidong Bing +2 more
This paper aims to solve the complete task of target-based sentiment analysis in an end-to-end fashion, and presents a novel unified model which applies a unified tagging scheme.
Proceedings of the AAAI Conference on Artificial IntelligenceEmpower Sequence Labeling with Task-Aware Neural Language Model
298 Citations2018Liyuan Liu, Jingbo Shang +5 more
This study develops a neural framework to extract knowledge from raw texts and empower the sequence labeling task, and leverages character-level knowledge from self-contained order information of training sequences.
Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks
293 Citations2019Binxuan Huang, Kathleen M. Carley
A novel target-dependent graph attention network (TD-GAT) is proposed for aspect level sentiment classification, which explicitly utilizes the dependency relationship among words to propagate sentiment features directly from the syntactic context of an aspect target.
An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis
272 Citations2019Ruidan He, Wee Sun Lee +2 more
An interactive multi-task learning network (IMN) is proposed which is able to jointly learn multiple related tasks simultaneously at both the token level as well as the document level and introduces a message passing architecture where information is iteratively passed to different tasks through a shared set of latent variables.
Adapting BERT for Target-Oriented Multimodal Sentiment Classification
261 Citations2019Jianfei Yu, Jing Jiang
This work studies Target-oriented Multimodal Sentiment Classification (TMSC) and proposes a multimodal BERT architecture and borrows the idea from selfattention and design a target attention mechanism to perform target-image matching to derive targetsensitive visual representations.
Attention Modeling for Targeted Sentiment
246 Citations2017Jiangming Liu, Yue Zhang
Results show that by using attention to model the contribution of each word in a sentence with respect to the target, this model gives significantly improved results over two standard benchmarks.
Neural Networks for Open Domain Targeted Sentiment
246 Citations2015Meishan Zhang, Yue Zhang +1 more
This work empirically studies the effect of word embeddings and automatic feature combinations on the open domain targeted sentiment task by extending a CRF baseline using neural networks, and proposes a novel integration of neural and discrete features which combines their relative advantages, leading to significantly higher results compared to both baselines.
Target-oriented Opinion Words Extraction with Target-fused Neural Sequence Labeling
231 Citations2019Zhifang Fan, Zhen Wu +3 more
This paper proposes a novel sequence labeling subtask for ABSA named TOWE (Target-oriented Opinion Words Extraction), which aims at extracting the corresponding opinion words for a given opinion target through a target-fused sequence labeling neural network model.
Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification
222 Citations2019Minghao Hu, Yuxing Peng +3 more
This work proposes a span-based extract-then-classify framework, where multiple opinion targets are directly extracted from the sentence under the supervision of target span boundaries, and corresponding polarities are then classified using their span representations.
Exploiting Document Knowledge for Aspect-level Sentiment Classification
219 Citations2018Ruidan He, Wee Sun Lee +2 more
Two approaches that transfer knowledge from document-level data, which is much less expensive to obtain, are explored to improve the performance of aspect-level sentiment classification.
Open Domain Targeted Sentiment
207 Citations2013Margaret Mitchell, Jacqui Aguilar +2 more
The intuition behind this work is that sentiment expressed towards an entity, targeted sentiment, may be viewed as a span of sentiment expressed across the entity, and this representation allows us to model sentiment detection as a sequence tagging problem, jointly discovering people and organizations along with whether there is sentiment directed towards them.
arXiv (Cornell University)Attentional Encoder Network for Targeted Sentiment Classification
197 Citations2019Youwei Song, Jiahai Wang +3 more
Proceedings of the AAAI Conference on Artificial IntelligenceLearning to Attend via Word-Aspect Associative Fusion for Aspect-Based Sentiment Analysis
196 Citations2018Yi Tay, Luu Anh Tuan +1 more
This paper proposes a novel method for integrating aspect information into the neural model by modeling word-aspect relationships and achieves state-of-the-art performance on benchmark datasets, outperforming ATAE-LSTM by 4%-5% on average across multiple datasets.
Target-Sensitive Memory Networks for Aspect Sentiment Classification
195 Citations2018Shuai Wang, Sahisnu Mazumder +3 more
The target-sensitive memory networks (TMNs) are proposed, which mean that the sentiment polarity of the (detected) context is dependent on the given target and it cannot be inferred from the context alone.
Parameterized Convolutional Neural Networks for Aspect Level Sentiment Classification
184 Citations2018Binxuan Huang, Kathleen M. Carley
A novel parameterized convolutional neural network is introduced for aspect level sentiment classification using parameterized filters and parameterized gates to incorporate aspect information into convolutionAL neural networks (CNN).
IEEE AccessDeep Learning for Aspect-Level Sentiment Classification: Survey, Vision, and Challenges
151 Citations2019Jie Zhou, Jimmy Xiangji Huang +4 more
A taxonomy of deep learning-based aspect-level sentiment classification is designed and a comprehensive summary of the state-of-the-art methods is given, showing the tremendous progress that has already been made in ASC.
arXiv (Cornell University)Empower Sequence Labeling with Task-Aware Neural Language Model
151 Citations2017Liyuan Liu, Jingbo Shang +5 more
DOER: Dual Cross-Shared RNN for Aspect Term-Polarity Co-Extraction
150 Citations2019Huaishao Luo, Tianrui Li +2 more
This paper proposes a novel Dual crOss-sharEd RNN framework (DOER) to generate all aspect term-polarity pairs of the input sentence simultaneously and demonstrates that the proposed framework outperforms state-of-the-art baselines on three benchmark datasets.
arXiv (Cornell University)Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification
132 Citations2019Alexander Rietzler, Sebastian Stabinger +2 more
This work approaches ATSC using a two-step procedure: self-supervised domain-specific BERT language model finetuned, followed by supervised task-specific finetuning, which enables it to produce new state-of-the-art performance on the SemEval 2014 Task 4 restaurants dataset.
IARM: Inter-Aspect Relation Modeling with Memory Networks in Aspect-Based Sentiment Analysis
130 Citations2018Navonil Majumder, Soujanya Poria +4 more
A novel approach of incorporating the neighboring aspects related information into the sentiment classification of the target aspect using memory networks and it is shown that this method outperforms the state of the art by 1.6% on average in two distinct domains: restaurant and laptop.
Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks
117 Citations2018Martin Schmitt, Simon Steinheber +2 more
The combination of a convolutional neural network and fasttext embeddings outperformed the best submission of the shared task in 2017, establishing a new state of the art for aspect-based sentiment analysis.
Proceedings of the AAAI Conference on Artificial IntelligenceExploiting Coarse-to-Fine Task Transfer for Aspect-Level Sentiment Classification
107 Citations2019Zheng Li, Ying Wei +3 more
This paper exploits a new direction named coarse-to-fine task transfer, which aims to leverage knowledge learned from a rich-resource source domain of the coarse-grained AC task, which is more easily accessible, to improve the learning in a low-resource target domain ofThe fine-grains AT task.
Joint Learning for Targeted Sentiment Analysis
71 Citations2018Dehong Ma, Sujian Li +1 more
Experimental results on two datasets show that the joint learning model can outperform other baselines and demonstrate the effectiveness of HSBi-GRU in learning abstract features.
Proceedings of the AAAI Conference on Artificial IntelligenceLearning Latent Sentiment Scopes for Entity-Level Sentiment Analysis
66 Citations2017Hao Li, Wei Lu
This paper proposes a novel approach that can explicitly model the latent sentiment scopes within which each named entity is embedded and is able to achieve better results compared to existing approaches based on conventional conditional random fields (CRFs) and a more recent work based on neural networks.
Proceedings of the AAAI Conference on Artificial IntelligenceA Human-Like Semantic Cognition Network for Aspect-Level Sentiment Classification
36 Citations2019Zeyang Lei, Yujiu Yang +4 more
A novel Human-like Semantic Cognition Network (HSCN) for aspect-level sentiment classification, motivated by the principles of human beings’ reading cognitive process, which achieves impressive results compared to other strong competitors.
Learning Explicit and Implicit Structures for Targeted Sentiment Analysis
26 Citations2019Hao Li, Wei Lu
This work argues that both types of information (implicit and explicit structural information) are crucial for building a successful targeted sentiment analysis model and shows that properly capturing both information is able to lead to better performance than competitive existing approaches.
arXiv (Cornell University)Learning Explicit and Implicit Structures for Targeted Sentiment\n Analysis
3 Citations2019Hao Li, Wei Lü