Speaker identification and video analysis for hierarchical video shot classification
Published 22 November 2002
Jeonghun Nam, A. Enis Çetin, Ahmed H. Tewfik
Citations44
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new video shot classification and clustering technique to support content-based indexing, browsing and retrieval in video databases based on the analysis of both the audio and visual data tracks.
Abstract
We present a new video shot classification and clustering technique to support content-based indexing, browsing and retrieval in video databases. The proposed method is based on the analysis of both the audio and visual data tracks. The visual stream is analyzed using a 3-D wavelet transform and segmented into shot units which are matched and clustered by visual content. Simultaneously, speaker changes are detected by tracking voiced phonemes in the audio signal. The clues obtained from the video and speech data are combined to classify and group the isolated video shots. This integrated approach also allows effective indexing of the audio-visual objects in multimedia databases.
Keywords
Computer Science
Fundamentals of speech recognition
7,687 Citations1993Lawrence R. Rabiner, Biing‐Hwang Juang
This book presents a meta-modelling framework for speech recognition that automates the very labor-intensive and therefore time-heavy and therefore expensive and expensive process of manually modeling speech.
Discrete-Time Processing of Speech Signals
2,462 Citations1999J.R. Deller, John H. L. Hansen +1 more
The preface to the IEEE Edition explains the background to speech production, coding, and quality assessment and introduces the Hidden Markov Model, the Artificial Neural Network, and Speech Enhancement.
IEEE Transactions on Circuits and Systems for Video TechnologyRapid scene analysis on compressed video
718 Citations1995Boon-Lock Yeo, Bede Liu
Experimental results show that the proposed rapid scene analysis algorithms are fast and effective in detecting abrupt scene changes, gradual transitions including fade-ins and fade-outs, flashlight scenes and in deriving intrashot variations.
Context-Dependent Phonetic Hidden Markov Models for Speaker-Independent Continuous Speech Recognition
213 Citations1992K.-F. Lee
Two new context-dependent phonetic units are introduced: function-word-dependent phone models, which focus on the most difficult subvocabulary; and generalized triphones, which combine similar triphones on the basis of an information-theoretic measure.
Multimedia SystemsAutomatic parsing and indexing of news video
212 Citations1995HongJiang Zhang, Shuang Yeo Tan +2 more
This paper's work on using domain knowledge to parse news video programs and to index them on the basis of their visual content is presented and experimental results are discussed in detail.
Proceedings - International Conference on Image ProcessingEfficient matching and clustering of video shots
203 Citations2002Minerva M. Yeung, Bede Liu
A scheme to match video shots and to cluster them by taking into account the temporal variations within individual shots is proposed, using a much reduced representation for a video shot.
Automatic parsing of news video
159 Citations1994Hao Zhang, Yihong Gong +2 more
Approaches to locating and identifying frame structure models based on temporal and spatial structure of news video data, along with algorithms to apply these models in parsing news video, have been developed and are presented in detail in this paper.
IEEE Transactions on Acoustics Speech and Signal ProcessingContext-independent phonetic hidden Markov models for speaker-independent continuous speech recognition
154 Citations1990K.-F. Lee
Integrated image and speech analysis for content-based video indexing
93 Citations2002Yuh-Lin Chang, Wenjun Zeng +2 more
The novelty of this work is that it proposes to integrate speech understanding and image analysis algorithms for extracting information in news or sports video indexing, where usually speech analysis is more efficient in detecting events than image analysis.
Audio as a support to scene change detection and characterization of video sequences
76 Citations2002C. Saraceno, Riccardo Leonardi
A technique which combines video and audio information together for classification and indexing purposes is presented, and a general framework that uses the results of such classification is proposed for organizing video information.
Combined audio and visual streams analysis for video sequence segmentation
48 Citations2002Jeonghun Nam, Ahmed H. Tewfik
This work presents a new approach to video sequence segmentation into individual shots by combining two streams of information extracted from the visual track with audio track segmentation information from a 3-D wavelet decomposition of the video track.
Subband analysis for robust speech recognition in the presence of car noise
48 Citations2002Engin Erzin, A. Enis Çetin +1 more
A new set of speech feature representations for robust speech recognition in the presence of car noise is proposed, based on subband analysis of the speech signal, and the performances of the new feature representations are compared to mel scale cepstral coefficients.
Frequency characteristics of foreign accented speech
33 Citations2002Levent M. Arslan, John H. L. Hansen
A new frequency scale is proposed in place of the commonly used Mel-scale to extract the cepstrum coefficients from the speech signal, which results in better performance for the problems of accent classification and language identification.
Video shot classification using human faces
17 Citations2002Yin Chan, Shang‐Hung Lin +2 more
A scheme to automatically detect the repeated occurrences of the same people to enable fast people related searching and a video shot classification scheme using human faces, regardless of scale and background is proposed.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Video classification using speaker identification</title>
15 Citations1997Nilesh Patel, Ishwar K. Sethi
This paper describes a three-stage processing system consisting of a shot boundary detection stage, an audio classification stage, and a speaker identification stage to determine the presence of different actors in isolated shots to show the efficacy of speaker identification for labeling video clips in terms of persons present in them.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Multichannel video segmentation</title>
6 Citations1996Pascal Faudemay, Liming Chen +6 more
A new approach of video segmentation into scenes is proposed, which is based on several media and takes into account the film syntax, and results are presented, based on experimental results.