Video data mining using con .gurations of viewpoint invariant regions
Published 12 November 2004
Josef Šivic, Andrew Zisserman
Citations152
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A method for obtaining the principal objects, characters and scenes in a video by measuring the reoccurrence of spatial configurations of viewpoint invariant features, and that efficient methods from the text analysis literature are employed to reduce the matching complexity.
Abstract
We describe a method for obtaining the principal objects, characters and scenes in a video by measuring the reoccurrence of spatial configurations of viewpoint invariant features. We investigate two aspects of the problem: the scale of the configurations, and the similarity requirements for clustering configurations. The problem is challenging firstly because an object can undergo substantial changes in imaged appearance throughout a video (due to viewpoint and illumination change, and partial occlusion), and secondly because configurations are detected imperfectly, so that inexact patterns must be matched. The novelty of the method is that viewpoint invariant features are used to form the configurations, and that efficient methods from the text analysis literature are employed to reduce the matching complexity. Examples of 'mined' objects are shown for a feature length film and a sitcom.
Keywords
Computer Science
Object recognition from local scale-invariant features
16,195 Citations1999David Lowe
Experimental results show that robust object recognition can be achieved in cluttered partially occluded images with a computation time of under 2 seconds.
Choice Reviews OnlineFinding groups in data: an introduction to cluster analysis
10,614 Citations1991
Foundations of statistical natural language processing
9,996 Citations1999Christopher D. Manning, Hinrich Schütze
Video Google: a text retrieval approach to object matching in videos
6,414 Citations2003Sivic, Zisserman
An approach to object and scene retrieval which searches for and localizes all the occurrences of a user outlined object in a video, represented by a set of viewpoint invariant region descriptors so that recognition can proceed successfully despite changes in viewpoint, illumination and partial occlusion.
Image and Vision ComputingRobust wide-baseline stereo from maximally stable extremal regions
3,664 Citations2004Jiřı́ Matas, Ondřej Chum +2 more
The high utility of MSERs, multiple measurement regions and the robust metric is demonstrated in wide-baseline experiments on image pairs from both indoor and outdoor scenes.
Object class recognition by unsupervised scale-invariant learning
2,035 Citations2003Rob Fergus, Pietro Perona +1 more
The flexible nature of the model is demonstrated by excellent results over a range of datasets including geometrically constrained classes (e.g. faces, cars) and flexible objects (such as animals).
IEEE Transactions on Pattern Analysis and Machine IntelligenceLocal grayvalue invariants for image retrieval
1,426 Citations1997C. Schmid, Roger Mohr
This paper addresses the problem of retrieving images from large image databases with a method based on local grayvalue invariants which are computed at automatically detected interest points and allows for efficient retrieval from a database of more than 1,000 images.
Lecture notes in computer scienceAn Affine Invariant Interest Point Detector
1,300 Citations2002Krystian Mikolajczyk, Cordelia Schmid
A novel approach for detecting affine invariant interest points that can deal with significant affine transformations including large scale changes and shows an excellent performance in the presence of large perspective transformations including significant scale changes.
Indexing based on scale invariant interest points
1,090 Citations2002Krystian Mikolajczyk, C. Schmid
The method is based on two recent results on scale space: interest points can be adapted to scale and give repeatable results (geometrically stable) and local extrema over scale of normalized derivatives indicate the presence of characteristic local structures.
A statistical method for 3D object detection applied to faces and cars
1,061 Citations2002Henry Schneiderman, Takeo Kanade
Using this method, this work has developed the first algorithm that can reliably detect human faces with out-of-plane rotation and the first algorithms thatCan reliably detect passenger cars over a wide range of viewpoints.
Lecture notes in computer scienceMulti-view Matching for Unordered Image Sets, or “How Do I Organize My Holiday Snaps?”
525 Citations2002F. Schaffalitzky, Andrew Zisserman
This paper invests how a combination of image invariants, covariants, and multiple view relations can be used in concord to enable efficient multiple view matching and produces a matching algorithm which is linear in the number of views.
Lecture notes in computer scienceLearning a Sparse Representation for Object Detection
513 Citations2002Shivani Agarwal, Dan Roth
An approach for learning to detect objects in still gray images, that is based on a sparse, part-based representation of objects, that achieves high detection accuracy on a difficult test set of real-world images, and is highly robust to partial occlusion and background variation.
Wide Baseline Stereo Matching based on Local, Affinely Invariant Regions
448 Citations2000Tinne Tuytelaars, Luc Van Gool
This work presents an alternative method for extracting invariant regions that does not depend on the presence of edges or corners in the image but is purely intensity-based, and demonstrates the use of such regions for another application, which is wide baseline stereo matching.
Constructing models for content-based image retrieval
363 Citations2005C. Schmid
A new method for constructing models from a set of positive and negative sample images; the method requires no manual extraction of significant objects or features and allows to efficiently capture "texture-like" visual structure.
International Journal of Image and GraphicsRELIABLE TRANSITION DETECTION IN VIDEOS: A SURVEY AND PRACTITIONER'S GUIDE
276 Citations2001Rainer Lienhart
This survey emphasizes those different core concepts underlying the different detection schemes for the three most widely used video transition effects: hard cuts, fades and dissolves.
Knowledge Discovery and Measures of Interest
222 Citations2001Robert J. Hilderman, Howard J. Hamilton
This paper presents a data mining technique and an interestingness framework based on heuristic measures of interestingness that were developed in the second part of this monograph on interestingness and data mining.
Object Recognition using Local Affine Frames on Distinguished Regions
201 Citations2002Štěpán Obdržálek, Jiřı́ Matas
A novel approach to appearance based object recognition based on matching of local image features, reliably recognises objects under very different viewing conditions that is invariant to piecewise-affine image deformations, but still remains very discriminative.
Lecture notes in computer scienceOn Affine Invariant Clustering and Automatic Cast Listing in Movies
135 Citations2002Andrew Fitzgibbon, Andrew Zisserman
It is demonstrated that the faces of the principal cast of a feature film can be generated automatically using clustering with appropriate invariance, and the affine invariant measure introduced may be obtained in closed form.
3D object modeling and recognition using affine-invariant patches and multi-view spatial constraints
127 Citations2003Fred Rothganger, Svetlana Lazebnik +2 more
Multi-view constraints associated with groups of patches are combined with a normalized representation of their appearance to guide matching and reconstruction, allowing the acquisition of true three-dimensional affine and Euclidean models from multiple images and their recognition in a single photograph taken from an arbitrary viewpoint.
Wide-baseline multiple-view correspondences
119 Citations2003Vittorio Ferrari, Tinne Tuytelaars +1 more
This paper presents a novel approach for establishing multiple-view feature correspondences along an unordered set of images taken from substantially different viewpoints, using an established WBS algorithm to extract and match features in pairs of views.
Affine-invariant local descriptors and neighborhood statistics for texture recognition
106 Citations2003Lazebnik, Schmid +1 more
A framework for texture recognition based on local affine-invariant descriptors and their spatial layout is presented and initial probabilities computed from the generative model are refined using a relaxation step that incorporates co-occurrence statistics.
Lecture notes in computer scienceVideo Summaries through Mosaic-Based Shot and Scene Clustering
90 Citations2002A. Aner, John R. Kender
A novel method for mosaic comparison is used and a highly compact non-temporal representation of video is created that allows accurate comparison of scenes across different videos and serves as a basis for indexing video libraries.
Lecture notes in computer scienceCombining Appearance and Topology for Wide Baseline Matching
79 Citations2002Dennis Tell, Stefan Carlsson
This paper incorporates topological constraints into an existing matching algorithm which matches image intensity profiles between interest points, and shows that the algorithm can be improved by exploiting the constraint that the intensity profiles around each interest point should be cyclically ordered.
Generating optimal video summaries
49 Citations2002Yihong Gong, Xin Liu
This work proposes a novel technique for video summarization based on singular value decomposition (SVD), which derives not only the refined feature space to better cluster visually similar frames, but also a metric to measure the amount of visual content contained in each frame cluster using its degree of visual change.
Content based indexing of images and video using face detection and recognition methods
44 Citations2002Stefan Eickeler, Frank Wallhoff +2 more
An image and video indexing approach that combines face detection and face recognition methods that is able to discriminate between three different newscasters and an interviewed person is presented.
Video personalization and summarization system
21 Citations2004B.L. Tseng, Ching‐Yung Lin +1 more
A video personalization and summarization system is designed and implemented to dynamically generate a personalized video summary that adopts the three-tier server-middleware-client architecture in order to select, adapt, and deliver rich media content to the user.