Scene Determination Based on Video and Audio Features
Multimedia Tools and ApplicationsPublished 1 September 2001
Silvia Pfeiffer, Rainer Lienhart, Wolfgang Efflsberg
Citations31
SJR quartileQ1
SJR score0.78
SNIP1.44
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A scheme for identifying scenes which clusters shots according to detected dialogs, settings and similar audio is developed and results from experiments show automatic identification of these types of scenes to be reliable.
Abstract
Determining automatically what constitutes a scene in a video is a challenging task, particularly since there is no precise definition of the term "scene". It is left to the individual to set attributes shared by consecutive shots which group them into scenes. Certain basic attributes such as dialogs, settings and continuing sounds are consistent indicators. We have therefore developed a scheme for identifying scenes which clusters shots according to detected dialogs, settings and similar audio. Results from experiments show automatic identification of these types of scenes to be reliable.
Keywords
Computer Science
IEEE Transactions on Neural NetworksFace recognition: a convolutional neural-network approach
3,090 Citations1997Sandra Lawrence, C. Lee Giles +2 more
A hybrid neural-network for human face recognition which compares favourably with other methods and analyzes the computational complexity and discusses how new classes could be added to the trained recognizer.
View-based and modular eigenspaces for face recognition
1,813 Citations1994Pentland, Moghaddam +1 more
A modular eigenspace description technique is used which incorporates salient features such as the eyes, nose and mouth, in an eigenfeature layer, which yields higher recognition rates as well as a more robust framework for face recognition.
Image indexing using color correlograms
1,658 Citations2002Jing Huang, S. Ravi Kumar +3 more
Experimental evidence suggests that this new image feature called the color correlogram outperforms not only the traditional color histogram method but also the recently proposed histogram refinement methods for image indexing/retrieval.
Film Art: An Introduction
894 Citations1979David Bordwell, Kristin Thompson
Comparing images using color coherence vectors
831 Citations1996Greg Pass, Ramin Zabih +1 more
It is shown that CCV’s can give superior results to color histogram-based methods for comparing images that incorporates spatial information, and to whom correspondence should be addressed tograms for image retrieval.
A feature-based algorithm for detecting and classifying scene breaks
491 Citations1995Ramin Zabih, Justin Miller +1 more
A new approach to the detection and classification of scene breaks in video sequences that can withstand compression artifacts such as those introduced by JPEG and MPEG, even at very high compression rates.
Digital Image Processing: Concepts, Algorithms, and Scientific Applications
383 Citations1991Bernd Jähne
Multimedia SystemsAutomatic parsing and indexing of news video
212 Citations1995HongJiang Zhang, Shuang Yeo Tan +2 more
This paper's work on using domain knowledge to parse news video programs and to index them on the basis of their visual content is presented and experimental results are discussed in detail.
The Journal of VLSI Signal Processing Systems for Signal Image and Video TechnologyAudio Feature Extraction and Analysis for Scene Segmentation and Classification
206 Citations1998Zhu Liu, Yao Wang +1 more
A set of low-level audio features are proposed for characterizing semantic contents of short audio clips and a neural net classifier was successful in separating the above five types of TV programs.
Content-based browsing of video sequences
193 Citations1994Farshid Arman, R. Depommier +2 more
The need to retrieve the video from a storage system and to transmit every frame over the network in its entirety no longer exists, saving time, expenses, and bandwidth.
Texture orientation for sorting photos "at a glance"
189 Citations2002M. Gorkani, Rosalind W. Picard
The authors demonstrate a simple test of global multiscale orientation for quickly searching a database of vacation photos for likely "city/suburb" shots and find the orientation features achieve agreement with human classification in 91 out of 98 of the scenes.
Time-constrained clustering for segmentation of video into story units
182 Citations1996Minerva M. Yeung, Boon-Lock Yeo
Time-constrained clustering of video shots is proposed to collapse visually similar and temporally local shots into a compact structure that allows the automatic segmentation of scenes and story units that cannot be achieved by existing shot boundary detection schemes.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Clustering methods for video browsing and annotation</title>
168 Citations1996Di Zhong, Hao Zhang +1 more
A generalized top-down hierarchial clustering process, which adopts partition clustering recursively at each level of the hierarchy, is studied and used to build hierarchical views of video shots.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Video browsing using clustering and scene transitions on compressed sequences</title>
156 Citations1995Minerva M. Yeung, Boon-Lock Yeo +2 more
A new technique for extracting a hierarchical decomposition of a complex video selection for browsing purposes that combines visual and temporal information to capture the important relations within a scene and between scenes in a video, thus allowing the analysis of the underlying story structure with no a priori knowledge of the content.
PanoramaExcerpts
102 Citations1997Yukinobu Taniguchi, Akihito Akutsu +1 more
PanoramaExcerpts is a video browsing interface that shows a catalogue of two types of video icons: panoramic and keyframe icons, which represents the entire visible contents of a scene extended with camera pan or tilt, which is difficult to summarize using a single keyframe.
Broadcast news navigation using story segmentation
94 Citations1997Andrew Merlino, Daryl Morey +1 more
This paper examines developed techniques and lessons learned in an operational multimedia exploitation system, Broadcast News Editor (BNE) and Broadcast News Navigator (BNN), which automatically processed over 6011 news stories from over 349 broadcasts of CNN Prime News.
Audio as a support to scene change detection and characterization of video sequences
76 Citations2002C. Saraceno, Riccardo Leonardi
A technique which combines video and audio information together for classification and indexing purposes is presented, and a general framework that uses the results of such classification is proposed for organizing video information.
IBM Journal of Research and DevelopmentVideo query: Research directions
70 Citations1998Ruud M. Bolle, Boon-Lock Yeo +1 more
A novel framework of structural video analysis that focuses on the processing of high-level features as well as low-level visual cues is presented that augments the semantic interpretation of a wide variety of long video segments and assists in the search, navigation, and retrieval of video.
A shot classification method of selecting effective key-frames for video browsing
68 Citations1996Hisashi Aoki, Shigeyoshi Shimotsuji +1 more
A method of reducing redundancy in key-frames by detecting this repetition using chromatic histograms and differences in local average luminance values is presented.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Video content characterization and compaction for digital library applications</title>
65 Citations1997Minerva M. Yeung, Boon-Lock Yeo
This paper develops models to capture and characterize video by temporal events, namely, dialogues, actions and story units, and presents these events using succinct visual summaries that depict and differentiate the underlying dramatic elements in an intuitive manner.
Locating and Tracking of Human Faces with Neural Networks
52 Citations1994H. M. Hunke
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>VisualGREP: a systematic method to compare and retrieve video sequences</title>
50 Citations1997Rainer Lienhart, Wolfgang Effelsberg +1 more
UC BerkeleyRecognition of Images in Large Databases Using a Learning Framework
48 Citations1997Serge Belongie, Chad Carson +2 more
A "blob world" representation is defined which provides a transition from the raw pixel data to a small set of localized coherent regions in color and texture space which is utilized to extract a probabilistic interpretation of the scene.
Speaker identification and video analysis for hierarchical video shot classification
44 Citations2002Jeonghun Nam, A. Enis Çetin +1 more
A new video shot classification and clustering technique to support content-based indexing, browsing and retrieval in video databases based on the analysis of both the audio and visual data tracks.
Vision
43 Citations1996Wei Li, Susan Gauch +2 more
The goal of the VISION (Video Indexing for SearchIng Over Networks) project is to establish a comprehensive, online digital videolibrary by developing automatic mechanisms to populate the library and provide content-based search and retrieval overcomputer networks.
National University of SingaporeClustering Methods for Video Browsing and Annotation
35 Citations1996Zhong, Di, Zhang, HongJiang +1 more
Enhanced video handling based on audio analysis
9 Citations1997Kenichi Minami, Akihito Akutsu +2 more
This paper proposes an audio-based approach to video indexing and handling that is implemented on a system called Video in Time as an example of creating reasonable condensed versions of dramas or movies by excerpting meaningful video segments.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Importance of perceptive adaptation of sound features in audio content processing</title>
5 Citations1998Silvia Pfeiffer
The results were used to evaluate two loudness measures: the sound pressure level, and an integral loudness measure, developed in the discussed research, which results in a significantly better approximation of the curve of human loudness estimates.
Towards a Visual Grep: A systematic analysis of various methods to compare video sequences
4 Citations1998Rainer Lienhart, Wolfgang Effelsberg +1 more
A general approach based on either a set or sequence representa-tion with variable degrees of aggregation is proposed and applied recursively over the different levels of temporal res-olution to allow the inquirer to fully control the importance of temporal ordering and duration.