A gaze prediction technique for open signed video content using a track before detect algorithm

Published 1 January 2008Open access

S.J.C. Davies, Dimitris K. Agrafiotis, C.N. Canagarajah, David Bull

DOI10.1109/icip.2008.4711852

Citations7

TL;DR

This paper proposes a gaze prediction model for open signed video content that is able to predict the results of an eye tracking study with up to 95% accuracy, and an average accuracy of over 80%.

Abstract

This paper proposes a gaze prediction model for open signed video content. A face detection algorithm is used to locate faces across each frame in both profile and frontal orientations. A grid-based likelihood ratio track before detect routine is used to predict the orientation of the signer's head, which allows the gaze location to be localised to either the signer or the inset. The face detections are then used to narrow down the gaze prediction further. The gaze predictor is able to predict the results of an eye tracking study with up to 95% accuracy, and an average accuracy of over 80%.

Keywords

Computer Science

International Journal of Computer VisionRobust Real-Time Face Detection

14,193 Citations2004Paul Viola, Michael Jones

A new image representation called the “Integral Image” is introduced which allows the features used by the detector to be computed very quickly and a method for combining classifiers in a “cascade” which allows background regions of the image to be quickly discarded while spending more computation on promising face-like regions.

CRC Press eBooksHandbook of multisensor data fusion

796 Citations2001David L. Hall, James Llinas

MULTISENSOR DATA FUSION Introduction to Multisensor Data Fusion, D. Hall and J.K. Bowman Introduction to the Algorithms of Data Association in Multiple-Target Tracking, and Studies and Analyses with Project Correlation: An In-Depth Assessment of Correlation Problems and Solution Techniques.

IEEE Transactions on Image ProcessingAutomatic Foveation for Video Compression Using a Neurobiological Model of Visual Attention

761 Citations2004L. Itti

A general-purpose usefulness of the algorithm is suggested in improving compression ratios of unconstrained video, based on a nonlinear integration of low-level visual cues, mimicking processing in primate occipital, and posterior parietal cortex.

The Journal of Deaf Studies and Deaf EducationPerception of Sign Language and Its Application to Visual Communications for Deaf People

119 Citations2005Laura Muir

Eye-movement tracking experiments were conducted with profoundly deaf volunteers while watching sign language video clips, finding that deaf people are found to fixate mostly on the facial region of the signer to pick up small detailed movements associated with facial expression and mouth shapes.

View PDF

Electronics LettersPerceptually optimised sign language video coding based on eye tracking analysis

46 Citations2003Dimitris K. Agrafiotis, Nishan Canagarajah +2 more

A perceptually optimised approach to sign language video coding based on the results of an eye tracking study in the visual attention of sign language viewers shows reductions in bit rate with very good subjective quality.

Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Optimized sign language video coding based on eye-tracking analysis</title>

21 Citations2003Dimitris K. Agrafiotis, C.N. Canagarajah +5 more

Analysis and results for two coding methods, one using MPEG-4 video objects and the second using foveation filtering are presented, offering a considerable decrease in bit rate in a way which is compatible with the visual attention patterns of deaf people.

Electronics Letters40 Gbit/s 16-bit label recognition using planar-lightwave-circuit-based all-optical serial-to-parallel converter

11 Citations2003Ryo Takahashi, Takashi Yasui +2 more

Electronics LettersPerceptually optimised coding of open signed video based on gaze prediction

2 Citations2007S.J.C. Davies, Dimitris K. Agrafiotis +2 more

A gaze location prediction model is presented for open signed broadcast video based on actual eye tracking data with fluent British sign language users and is shown to be capable of offering up to 30% reduction in bit rate.