Audio-visual based emotion recognition using tripled hidden Markov model

Published 28 September 2004

Mingli Song, Chun Chen, Mingyu You

DOI10.1109/icassp.2004.1327251

Citations28

TL;DR

A novel approach is presented in this paper to recognize the human emotion which uses both visual and audio from video clips and a tripled hidden Markov model is introduced to perform the recognition.

Abstract

Emotion recognition is one of the latest challenges in intelligent human/machine communication. Most of previous work on emotion recognition focused on extracting emotions from visual or audio information separately. A novel approach is presented in this paper to recognize the human emotion which uses both visual and audio from video clips. A tripled hidden Markov model is introduced to perform the recognition which allows the state asynchrony of the audio and visual observation sequences while preserving their natural correlation over time. The experimental results show that this approach outperforms only using visual or audio separately.

Keywords

PsychologyComputer Science

Lecture notes in computer scienceActive appearance models

2,196 Citations1998T.F. Cootes, G. Edwards +1 more

A new method of matching statistical models of appearance to images by learning the relationship between perturbations in the model parameters and the induced image errors is described.

Psychological BulletinStrong evidence for universals in facial expressions: A reply to Russell's mistaken critique.

1,053 Citations1994Paul Ekman

The evidence from both literate and preliterate cultures is overwhelming in support of universals in facial expressions, and A.J. Russell's qualms about other aspects of the design of the studies of literate cultures have no merit.

Computer Vision and Image UnderstandingFacial expression recognition from video sequences: temporal and static modeling

873 Citations2003Ira L. Cohen, Nicu Sebe +3 more

This work introduces and test different Bayesian network classifiers for classifying expressions from video, focusing on changes in distribution assumptions, and feature dependency structures, and proposes a new architecture of hidden Markov models (HMMs) for automatically segmenting and recognizing human facial expression from video sequences.

Recognizing emotion in speech

498 Citations2002Frank Dellaert, Thomas Polzin +1 more

A new method of extracting prosodic features from speech, based on a smoothing spline approximation of the pitch contour, is presented, which obtains classification performance that is close to human performance on the task.

View PDF

Speech emotion recognition using hidden Markov models

231 Citations2001Albino Nogueiras Rodríguez, Asunción Moreno +2 more

This paper introduces a first approach to emotion recognition using RAMSES, the UPC’s speech recognition system, based on standard speech recognition technology using hidden semi-continuous Markov models.

Learning Bayesian network classifiers for facial expression recognition both labeled and unlabeled data

156 Citations2003Ira L. Cohen, Nicu Sebe +3 more

A classification driven stochastic structure search algorithm for learning the structure of Bayesian network classifiers is introduced, and it is shown that with moderate size labeled training sets and large amount of unlabeled data, the method can utilize unlabeling data to improve classification performance.

Eigen-points [image matching]

23 Citations2002Michele Covell, Christoph Bregler

Eigen-points places control points onto unmarked images using a coupled manifold model, which describes the joint variation of the image appearance and the control-point location.