Emotion recognition using semi-supervised feature selection with speaker normalization
International Journal of Speech TechnologyPublished 3 February 2015
Yaxin Sun, Guihua Wen
Citations18
SJR quartileQ1
SJR score0.38
SNIP0.98
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A semi-supervised feature selection method that can preserve the manifold structure of data, preserve the category structure, and use the information provided by the unlabeled data is presented and a new speaker normalization method is proposed, which can achieve a good speakernormalization result in the case of a small number of samples of a speaker available.
Abstract
Feature selection methods are the mostly used dimensional reduction methods in speech emotion recognition. However, most methods cannot preserve the manifold of data and cannot use the information provided by unlabeled data, so that they cannot select a good sub feature set for speech emotion recognition. This paper presents a semi-supervised feature selection method that can preserve the manifold structure of data, preserve the category structure, and use the information provided by the unlabeled data. To further deal with the manifold of speech data influenced by factors such as emotion, speaker and sentence, a new speaker normalization method is also proposed, which can achieve a good speaker normalization result in the case of a small number of samples of a speaker available. This speaker normalization method can be used in most real application of speech emotion recognition. The conducted experiments validate the proposed semi-supervised feature selection method with the speaker normalization in terms of the effectiveness of the speech emotion recognition.
Keywords
PsychologyComputer Science
ACM Transactions on Intelligent Systems and TechnologyLIBSVM
41,340 Citations2011Chih-Chung Chang, Chih‐Jen Lin
Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
ScienceNonlinear Dimensionality Reduction by Locally Linear Embedding
15,035 Citations2000Sam T. Roweis, Lawrence K. Saul
Locally linear embedding (LLE) is introduced, an unsupervised learning algorithm that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional inputs that learns the global structure of nonlinear manifolds.
IEEE Transactions on Pattern Analysis and Machine IntelligenceFeature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy
10,470 Citations2005Hanchuan Peng, Fuhui Long +1 more
This work derives an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection, and presents a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers).
Neural ComputationLaplacian Eigenmaps for Dimensionality Reduction and Data Representation
7,709 Citations2003Mikhail Belkin, Partha Niyogi
This work proposes a geometrically motivated algorithm for representing the high-dimensional data that provides a computationally efficient approach to nonlinear dimensionality reduction that has locality-preserving properties and a natural connection to clustering.
Pattern Recognition LettersFloating search methods in feature selection
3,070 Citations1994Pavel Pudil, Jana Novovičová +1 more
A database of German emotional speech
2,200 Citations2005Felix Burkhardt, Astrid Paeschke +3 more
A database of emotional speech that was evaluated in a perception test regarding the recognisability of emotions and their naturalness and can be accessed by the public via the internet.
Pattern RecognitionSurvey on speech emotion recognition: Features, classification schemes, and databases
1,970 Citations2010Moataz El Ayadi, Mohamed S. Kamel +1 more
A survey of speech emotion classification addressing three important aspects of the design of a speech emotion recognition system, the choice of suitable features for speech representation, and the proper preparation of an emotional speech database for evaluating system performance are addressed.
CaltechAUTHORS (California Institute of Technology)Self-Tuning Spectral Clustering
1,913 Citations2004Lihi Zelnik‐Manor, Pietro Perona
This work proposes that a 'local' scale should be used to compute the affinity between each pair of points and suggests exploiting the structure of the eigenvectors to infer automatically the number of groups.
Laplacian Score for Feature Selection
1,768 Citations2005Xiaofei He, Deng Cai +1 more
This paper proposes a "filter" method for feature selection which is independent of any learning algorithm, based on the observation that, in many real world classification problems, data from the same class are often close to each other.
Unsupervised feature selection for multi-cluster data
1,113 Citations2010Deng Cai, Chiyuan Zhang +1 more
Inspired from the recent developments on manifold learning and L1-regularized models for subset selection, a new approach is proposed, called Multi-Cluster Feature Selection (MCFS), for unsupervised feature selection, which select those features such that the multi-cluster structure of the data can be best preserved.
The INTERSPEECH 2009 emotion challenge
911 Citations2009Björn W. Schuller, Stefan Steidl +1 more
The challenge, the corpus, the features, and benchmark results of two popular approaches towards emotion recognition from speech, and the FAU Aibo Emotion Corpus are introduced.
Speech CommunicationSpeech emotion recognition using hidden Markov models
907 Citations2003Tin Lay Nwe, Say Wei Foo +1 more
This paper proposes a text independent method of emotion classification of speech that makes use of short time log frequency power coefficients (LFPC) to represent the speech signals and a discrete hidden Markov model (HMM) as the classifier.
Spectral feature selection for supervised and unsupervised learning
904 Citations2007Zheng Zhao, Huan Liu
This work exploits intrinsic properties underlying supervised and unsupervised feature selection algorithms, and proposes a unified framework for feature selection based on spectral graph theory, and shows that existing powerful algorithms such as ReliefF and Laplacian Score are special cases of the proposed framework.
The INTERSPEECH 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism
724 Citations2013Björn W. Schuller, Stefan Steidl +13 more
The INTERSPEECH 2013 Computational Paralinguistics Challenge provides for the first time a unified test-bed for Social Signals such as laughter in speech and introduces conflict in group discussions as a new task and deals with autism and its manifestations in speech.
The INTERSPEECH 2010 paralinguistic challenge
562 Citations2010Björn W. Schuller, Stefan Steidl +5 more
The INTERSPEECH 2010 Paralinguistic Challenge shall help overcome the usually low compatibility of results, by addressing three selected sub-challenges, by address-ing three selected tasks.
IEEE Transactions on CyberneticsJoint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection
559 Citations2013Chenping Hou, Feiping Nie +3 more
This paper proposes a novel unsupervised feature selection framework, termed as the joint embedding learning and sparse regression (JELSR), in which the embedding learned with sparse regression to perform feature selection.
IEEE Transactions on Affective ComputingCross-Corpus Acoustic Emotion Recognition: Variances and Strategies
381 Citations2010Björn W. Schuller, Bogdan Vlasenko +5 more
Results employing six standard databases in a cross-corpora evaluation experiment show the crucial performance inferiority of inter to intracorpus testing and investigates different types of normalization.
Speech CommunicationAutomatic speech emotion recognition using modulation spectral features
381 Citations2010Siqing Wu, Tiago H. Falk +1 more
Modulation spectral features are proposed for the automatic recognition of human affective information from speech and render a substantial improvement in recognition performance when used to augment prosodic features, which have been extensively used for emotion recognition.
Local Fisher discriminant analysis for supervised dimensionality reduction
371 Citations2006Masashi Sugiyama
A new dimensionality reduction method called local Fisher discriminant analysis (LFDA) is proposed, which is a localized variant of Fisher discriminating analysis that takes local structure of the data into account so the multimodal data can be embedded appropriately.
Speech CommunicationEmotion recognition using a hierarchical binary decision tree approach
354 Citations2011Chi-Chun Lee, Emily Mower +3 more
A hierarchical computational structure to recognize emotions is introduced that maps an input speech utterance into one of the multiple emotion classes through subsequent layers of binary classifications and is effective for classifying emotional utterances in multiple database contexts.
Floating search methods for feature selection with nonmonotonic criterion functions
351 Citations2002Pavel Pudil, Francesc J. Ferri +2 more
The recently developed "floating search" algorithms are presented and modified to a more compact form facilitating their direct comparison with the well known (l,r) search.
Proceedings of the AAAI Conference on Artificial IntelligenceEfficient Spectral Feature Selection with Minimum Redundancy
282 Citations2010Zheng Zhao, Lei Wang +1 more
This work proposes a novel spectral feature selection algorithm to handle feature redundancy, adopting an embedded model derived from a formulation based on a sparse multi-output regression with a L2,1-norm constraint.
Journal of the Royal Statistical Society Series C (Applied Statistics)Selection of Variables to Preserve Multivariate Data Structure, Using Principal Components
274 Citations1987W. J. Krzanowski
IEEE Transactions on Affective ComputingEmotion Recognition of Affective Speech Based on Multiple Classifiers Using Acoustic-Prosodic Information and Semantic Labels
266 Citations2010Chung‐Hsien Wu, Wei-Bin Liang
This work presents an approach to emotion recognition of affective speech based on multiple classifiers using acoustic-prosodic information (AP) and semantic labels (SLs) and reveals that the recognition accuracy of the proposed approach can be further improved to 85.79 percent.
Digital Signal ProcessingSpeech emotion recognition: Features and classification models
228 Citations2012Lijiang Chen, Xia Mao +2 more
The experimental results proved that Fisher is better than PCA for dimension reduction, and SVM is more expansible than ANN for speaker independent speech emotion recognition.
Speech CommunicationClass-level spectral features for emotion recognition
218 Citations2010Dmitri Bitouk, Ragini Verma +1 more
This work introduces a more fine-grained yet robust set of spectral features: statistics of Mel-Frequency Cepstral Coefficients computed over three phoneme type classes of interest-stressed vowels, unstressed vowel and consonants in the utterance.
The INTERSPEECH 2011 speaker state challenge
217 Citations2011Björn W. Schuller, Stefan Steidl +3 more
The INTERSPEECH 2011 Speaker State Challenge addresses two new sub-challenges to overcome the usually low compatibility of results: in the Intoxication Sub-Challenge, alcoholisation of speakers has to be determined in two classes; in the Sleepy Language Corpus, another two-class classification task has to been solved.
The INTERSPEECH 2012 speaker trait challenge
214 Citations2012Björn W. Schuller, Stefan Steidl +10 more
The EPFL-CONF-174360 data indicate that speaker Traits and Likability are influenced by the environment and the speaker’s personality in terms of paralinguistics and personality.
Information Processing & ManagementAcoustic feature selection for automatic emotion recognition from speech
209 Citations2008Jia Rong, Gang Li +1 more
A novel algorithm is presented in this paper, which can be applied on a small sized data set with a high number of features and outperform the commonly used Principle Component Analysis (PCA)/Multi-Dimensional Scaling (MDS) methods, and the more recently developed ISOMap dimensionality reduction method.
Lecture notes in computer scienceOn the Use of Variable Complementarity for Feature Selection in Cancer Classification
166 Citations2006Patrick Meyer, Gianluca Bontempi
An original filter approach for effective feature selection in classification tasks with a very large number of input variables based on a new information theoretic selection criterion: the double input symmetrical relevance (DISR).
IEEE Transactions on MultimediaFeature Analysis and Evaluation for Automatic Emotion Identification in Speech
160 Citations2010Iker Luengo, Eva Navas +1 more
Analysis of the characteristics of features derived from prosody, spectral envelope, and voice quality as well as their capability to discriminate emotions suggest that spectral envelope features outperform the prosodic ones.
Computer Speech & LanguageSpoken emotion recognition using hierarchical classifiers
138 Citations2010Enrique M. Albornoz, Diego H. Milone +1 more
The spectral characteristics of emotional signals are used in order to group emotions based on acoustic rather than psychological considerations, and the proposed multiple feature hierarchical method for seven emotions improves the performance over the standard classifiers and the fixed features.
Pattern RecognitionA supervised non-linear dimensionality reduction approach for manifold learning
136 Citations2011Bogdan Raducanu, Fadi Dornaika
A novel supervised manifold learning technique called Supervised Laplacian Eigenmaps (S-LE), which makes use of class label information to guide the procedure of non-linear dimensionality reduction by adopting the large margin concept.
Speech CommunicationAn evaluation of the robustness of existing supervised machine learning approaches to the classification of emotions in speech
131 Citations2007Mohammad Shami, Werner Verhelst
The robustness of approaches to the automatic classification of emotions in speech is addressed and it is suggested that existing approaches are efficient enough to handle larger amounts of training data without any reduction in classification accuracy.
Surrey Open Research repository (University of Surrey)Speaker-dependent audio-visual emotion recognition
128 Citations2009Sanaul Haq, Philip J. B. Jackson
The results were comparable to the measured human recognition rate with this multimodal data set and the performance was higher for LDA features compared to PCA features.
IEEE Transactions on Consumer ElectronicsFeature vector classification based speech emotion recognition for service robots
99 Citations2009Jeong‐Sik Park, Ji‐Hwan Kim +1 more
On SER experiments using an emotional speech corpus, the proposed classification approach exhibited superior performance to conventional methods, and displayed an almost human-level performance, achieved commercially applicable performance for two-class (negative vs. non-negative) emotion recognition.
Neural Computing and ApplicationsSpeech emotion recognition using FCBF feature selection method and GA-optimized fuzzy ARTMAP neural network
94 Citations2011Davood Gharavian, Mansour Sheikhan +2 more
Experimental results show the improvement in emotion recognition rate of angry, happiness, and neutral states by using a subset of 25 selected features and the GA-optimized FAMNN-based emotion recognizer.
IEEE Transactions on Affective ComputingModeling the Temporal Evolution of Acoustic Parameters for Speech Emotion Recognition
88 Citations2011Stavros Ntalampiras, Nikos Fakotakis
A series of exhaustive experiments are described which demonstrate the feasibility of recognizing human emotional states via integrating low level descriptors via integrating three different methodologies for integrating subsequent feature values.
Lecture notes in computer scienceFrame vs. Turn-Level: Emotion Recognition from Speech Considering Static and Dynamic Processing
86 Citations2007Bogdan Vlasenko, Björn W. Schuller +2 more
This work re-investigate dynamic modeling directly on the frame-level in speech-based emotion recognition and integrates this frame- level information within a state-of-the-art large-feature-space emotion recognition engine.
Computer Speech & LanguageSpoken emotion recognition through optimum-path forest classification using glottal features
84 Citations2009Alexander I. Iliev, Michael S. Scordilis +2 more
Experimental results indicate that best performance is obtained for theglottal-only features with SVM and OPF generally providing the highest recognition rates, while for GMM or the combination of glottal and speech features performance was relatively inferior.
Pattern RecognitionTrace ratio criterion based generalized discriminative learning for semi-supervised dimensionality reduction
75 Citations2011Mingbo Zhao, Zhao Zhang +1 more
The proposed algorithm incorporates unlabeled data into training procedure so that it is able to preserve the discriminative structure as well as geometrical structure embedded in the original dataset and can achieve great superiority to other state-of-art algorithms.
Biomedical Signal Processing and ControlStudy of empirical mode decomposition and spectral analysis for stress and emotion classification in natural speech
65 Citations2010Ling He, Margaret Lech +2 more
Two new approaches to the feature extraction process for automatic stress and emotion classification in speech are proposed and examined and indicated that the spectrogram patterns provide promising results in the cases of stress recognition, however further improvements are needed in the case of emotion recognition.
Speech CommunicationFormant position based weighted spectral features for emotion recognition
63 Citations2011Elif Bozkurt, Engin Erzin +2 more
This paper proposes novel spectrally weighted mel-frequency cepstral coefficient (WMFCC) features for emotion recognition from speech, and evaluates the proposed WMFCC features together with the standard spectral and prosody features using HMM based classifiers on the spontaneous FAU Aibo emotional speech corpus.
Iterative feature normalization for emotional speech detection
54 Citations2011Carlos Busso, Angeliki Metallinou +1 more
This paper introduces a feature normalization scheme that implements these ideas by iteratively detecting neutral speech and normalizing the features, and as the approximation error of the normalization parameters is reduced, the accuracy of the emotion detection system increases.
Pattern RecognitionSemi-supervised metric learning via topology preserving multiple semi-supervised assumptions
52 Citations2013Qianying Wang, Pong C. Yuen +1 more
Experimental results have shown that the proposed method outperforms state-of-the-art semi-supervised distance metric learning algorithms.
Speaker Normalisation for Speech-Based Emotion Detection
51 Citations2007Vidhyasaharan Sethu, Eliathamby Ambikairajah +1 more
This paper compares the performance of a system that uses feature warping to one that does not, and proposes the use of speaker-specific feature warped as a means of normalising acoustic features to overcome the problem of speaker dependency.
Computers in Human BehaviorSegment-based emotion recognition from continuous Mandarin Chinese speech
48 Citations2010Jun-Heng Yeh, Tsang-Long Pao +3 more
A segment-based emotion recognition approach to continuous Mandarin Chinese speech with an emotion radar chart based on WD-KNN that shows how emotions can be recognized by speech signals, and in turn how emotional states can be visualized.
IEEE Transactions on Audio Speech and Language ProcessingLoss-Scaled Large-Margin Gaussian Mixture Models for Speech Emotion Classification
46 Citations2011Sungrack Yun, Chang D. Yoo
Results consistently show that margin scaling improves the classification accuracy over other learning frameworks based on the maximum-likelihood, maximum mutual information and max-margin framework without margin scaling.
Speech CommunicationClassification of emotional speech using 3DEC hierarchical classifier
42 Citations2012Ali Hassan, R.I. Damper
A new hierarchical classification technique whose structure is based on NMDS, which is called Data-Driven Dimensional Emotion Classification (3DEC), which significantly outperforms the competitors for the much more interesting and important case of speaker-independent emotion classification.
Future Generation Computer SystemsEnhanced semi-supervised local Fisher discriminant analysis for face recognition
39 Citations2010Hong Huang, Jianwei Li +1 more
An improved manifold learning method, called enhanced semi-supervised local Fisher discriminant analysis (ESELF), for face recognition is proposed, which has an analytic form of the globally optimal solution and it can be computed based on eigen decomposition.
Biomedical Signal Processing and ControlAcoustic feature selection and classification of emotions in speech using a 3D continuous emotion model
37 Citations2011Humberto Pérez-Espinosa, Carlos A. Reyes-García +1 more
The feasibility of applying the continuous emotion models approach to annotation of emotional speech is demonstrated and ways to take advantage of this kind of annotation to improve the automatic classification of basic emotions are explored.
Multimedia Tools and ApplicationsDimensionality reduction-based spoken emotion recognition
29 Citations2011Shiqing Zhang, Xiaoming Zhao
A new supervised manifold learning algorithm for nonlinear dimensionality reduction, called modified supervised locally linear embedding algorithm (MSLLE) is proposed for spoken emotion recognition, which aims at enlarging the interclass distance while shrinking the intraclass distance in an effort to promote the discriminating power and generalization ability of low-dimensional embedded data representations.
Speaker variability in emotion recognition - an adaptation based approach
25 Citations2012Ni Ding, Vidhyasaharan Sethu +2 more
A novel bootstrapping technique which involves selecting appropriate initial models from a large training pool, prior to speaker adaptation of emotion models in the context of GMM based emotion classification is proposed as an alternative to speaker normalisation.
Recognition of emotions in speech by a hierarchical approach
25 Citations2009Zhongzhe Xiao, Emmanuel Dellandréa +2 more
Some new harmonic and Zipf based features for better speech emotion characterization in the valence dimension and a multistage classification scheme driven by a dimensional emotion model for better emotional class discrimination are proposed.
Speech CommunicationEnhancement of emotion detection in spoken dialogue systems by combining several information sources
23 Citations2011Ramón López-Cózar, Jan Silovský +1 more
The results show that the technique enhances the classification rates of the standard fusion by 2.27% and 3.38% absolute in experiments carried out considering two and three emotion categories, respectively.
Pattern RecognitionSemi-supervised learning with nuclear norm regularization
19 Citations2013Fanhua Shang, Licheng Jiao +2 more
This paper proposes a novel semi-supervised learning approach, called semi- supervised learning with nuclear norm regularization (SSL-NNR), which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data.
Feature extraction based on zero-crossings with peak amplitudes for robust speech recognition in noisy environments
13 Citations2002Doh-Suk Kim, Jaehoon Jeong +2 more
An analytic relationship is developed for the variance and SNR of the level-crossing intervals as a function of the crossing level value, and a new feature extraction method based on zero-crossings with peak amplitudes is proposed for robust speech recognition in noisy environments.
Optics CommunicationsSemi-supervised kernel learning based optical image recognition
4 Citations2012Jun-Bao Li, Zhiming Yang +2 more
This paper proposes semi-supervised kernel learning based optical image recognition through integrating graph construction with the specific DR process into one unified framework called SGGLPP, which preserves not only the positive and negative constraints but also the local and global structure of the data in the low dimensional space.