Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-level Multimodal Sentiment Analysis
Published 1 January 2015Open access
Soujanya Poria, Erik Cambria, Alexander Gelbukh
Citations534
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A novel way of extracting features from short texts, based on the activation values of an inner layer of a deep convolutional neural network, is presented and a parallelizable decision-level data fusion method is presented, which is much faster, though slightly less accurate.
Abstract
We present a novel way of extracting features from short texts, based on the activation values of an inner layer of a deep convolutional neural network. We use the extracted features in multimodal sentiment analysis of short video clips representing one sentence each. We use the combined feature vectors of textual, visual, and audio modalities to train a classifier based on multiple kernel learning, which is known to be good at heterogeneous data. We obtain 14% performance improvement over the state of the art and present a parallelizable decision-level data fusion method, which is much faster, though slightly less accurate.
Keywords
Computer Science
arXiv (Cornell University)Distributed Representations of Words and Phrases and their Compositionality
18,086 Citations2013Tomáš Mikolov, Ilya Sutskever +3 more
This paper presents a simple method for finding phrases in text, and shows that learning good vector representations for millions of phrases is possible and describes a simple alternative to the hierarchical softmax called negative sampling.
arXiv (Cornell University)Efficient Estimation of Word Representations in Vector Space
18,050 Citations2013Tomáš Mikolov, Kai Chen +2 more
Two novel model architectures for computing continuous vector representations of words from very large data sets are proposed and it is shown that these vectors provide state-of-the-art performance on the authors' test set for measuring syntactic and semantic word similarities.
Convolutional Neural Networks for Sentence Classification
13,798 Citations2014Yoon Kim
The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification, and are proposed to allow for the use of both task-specific and static vectors.
Linguistic Regularities in Continuous Space Word Representations
2,882 Citations2013Tomáš Mikolov, Wen-tau Yih +1 more
The vector-space word representations that are implicitly learned by the input-layer weights are found to be surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset.
Opensmile
2,558 Citations2010Florian Eyben, Martin Wöllmer +1 more
The openSMILE feature extraction toolkit is introduced, which unites feature extraction algorithms from the speech processing and the Music Information Retrieval communities and has a modular, component based architecture which makes extensions via plug-ins easy.
NeurocomputingFusing audio, visual and textual clues for sentiment analysis from multimodal content
524 Citations2015Soujanya Poria, Erik Cambria +3 more
This paper proposes a novel methodology for multimodal sentiment analysis, which consists in harvesting sentiments from Web videos by demonstrating a model that uses audio, visual and textual modalities as sources of information.
Towards multimodal sentiment analysis
441 Citations2011Louis–Philippe Morency, Rada Mihalcea +1 more
This paper addresses the task of multimodal sentiment analysis, and conducts proof-of-concept experiments that demonstrate that a joint model that integrates visual, audio, and textual features can be effectively used to identify sentiment in Web videos.
Computación y SistemasSoft Similarity and Soft Cosine Measure: Similarity of Features in Vector Space Model
369 Citations2014Grigori Sidorov, Alexander Gelbukh +2 more
The proposed similarity measure soft similarity is a generalize of the well-known cosine similarity measure in VSM by introducing what it is called “soft cosine measure” and various formulas for exact or approximate calculation of the softcosine measure are proposed.
Knowledge-Based SystemsSentic patterns: Dependency-based rules for concept-level sentiment analysis
342 Citations2014Soujanya Poria, Erik Cambria +2 more
A novel paradigm to concept-level sentiment analysis is introduced that merges linguistics, common-sense computing, and machine learning for improving the accuracy of tasks such as polarity detection.
3D Constrained Local Model for rigid and non-rigid facial tracking
313 Citations2012Tadas Baltrušaitis, Peter Robinson +1 more
A way to combine a rigid head pose tracker with CLM-Z that benefits rigid head tracking and shows better performance than the current state-of-the-art approaches in head pose tracking with the extension of the generalised adaptive view-based appearance model (GAVAM).
Utterance-Level Multimodal Sentiment Analysis
241 Citations2013Verónica Pérez‐Rosas, Rada Mihalcea +1 more
It is shown that multimodal sentiment analysis can be effectively performed, and that the joint use of visual, acoustic, and linguistic modalities can lead to error rate reductions of up to 10.5% as compared to the best performing individual modality.
Neural NetworksTowards an intelligent framework for multimodal affective data analysis
227 Citations2014Soujanya Poria, Erik Cambria +2 more
A novel multimodal information extraction agent is proposed, which infers and aggregates the semantic and affective information associated with user-generated multi-modal data in contexts such as e-learning, e-health, automatic video content tagging and human-computer interaction.
Knowledge-Based SystemsEmoSenticSpace: A novel framework for affective common-sense reasoning
170 Citations2014Soujanya Poria, Alexander Gelbukh +3 more
EmoSenticSpace, a new framework for affective common-sense reasoning that extends WordNet-Affect and SenticNet by providing both emotion labels and polarity scores for a large set of natural language concepts, is proposed.
Sentic Computing: A Common-Sense-Based Framework for Concept-Level Sentiment Analysis
161 Citations2015Erik Cambria, Amir Hussain
This chapter proposes insights on the evolution of opinion mining research from heuristics to discourse structure, from coarseto fine-grained analysis, and from keyword to concept-level polarity detection and introduces sentic computing as a new common-sense based framework for concept- level sentiment analysis.
IEEE Intelligent SystemsKnowledge-Based Approaches to Concept-Level Sentiment Analysis
132 Citations2013Erik Cambria, Björn W. Schuller +3 more
The guest editors introduce novel approaches to opinion mining and sentiment analysis that go beyond a mere word-level analysis of text and provide concept-level methods that allow a more efficient passage from (unstructured) textual information to machine-processable data, in potentially any domain.
Knowledge-Based SystemsGuest Editorial: Big Social Data Analysis
122 Citations2014Erik Cambria, Haixun Wang +1 more
The contained articles in this issue cover variegated topics including stock market prediction, political forecasting, time-evolving opinion mining, social network analysis, and human–robot interaction.
Sentic blending: Scalable multimodal fusion for the continuous interpretation of semantics and sentics
103 Citations2013Erik Cambria, Newton Howard +2 more
This work proposes a methodology that enables the continuous interpretation of semantics and sentics based on the integration of an affective common-sense knowledge base with any multimodal signal-processing module, interfaced with a facial emotional classifier and an opinion mining engine.
IEEE Intelligent SystemsStatistical Approaches to Concept-Level Sentiment Analysis
101 Citations2013Erik Cambria, Björn W. Schuller +3 more
The guest editors introduce novel statistical approaches to concept-level sentiment analysis that go beyond a mere syntactic-driven analysis of text and provide semantic-based methods that allow a more efficient passage from (unstructured) textual information to machine-processable data, in potentially any domain.
Feature Extraction with Convolutional Neural Networks for Handwritten Word Recognition
78 Citations2013Théodore Bluche, Hermann Ney +1 more
It is shown that learning features with convolutional neural networks is better than using hand-crafted features for handwritten word recognition by comparison with a state-of-the art HMM system based on explicit feature extraction.
Cognitive ComputationCommon Sense Knowledge for Handwritten Chinese Text Recognition
78 Citations2012Qiufeng Wang, Erik Cambria +2 more
Three methods are introduced to supplement the standard n-gram language model and the results show that the adoption of common sense knowledge yields improvements in recognition performance, despite the reduced concept list hereby employed.
SPF-GMKL
76 Citations2012Ashesh Jain, S. V. N. Vishwanathan +1 more
A Spectral Projected Gradient descent optimizer is developed which takes into account second order information in selecting step sizes, employs a non-monotone step size selection criterion requiring fewer function evaluations, is robust to gradient noise, and can take quick steps when far away from the optimum.
Theory and applications of categoriesA Textual Entailment System using Anaphora Resolution.
19 Citations2011Partha Pakray, Snehasis Neogi +4 more
The note describes the Recognizing Textual Entailment (RTE) system developed at the Computer Science and Engineering Department, Jadavpur University, India, which is based on pre-processing task which includes Anaphora Resolution using JavaRAP tool.
PolibitsSoft Cardinality in Semantic Text Processing: Experience of the SemEval International Competitions
14 Citations2015Sergio Jiménez, Fabio A. González +1 more
This paper describes the experience of using soft cardinality to address a series of natural language processing (NLP) tasks in the recent SemEval competitions from 2012 to 2014, and presents the generalities of the model and some practical techniques for usingsoft cardinality for NLP problems.