On the synthesis of dynamic scenes from reference views

Published 7 November 2002

Yonatan Wexler, Amnon Shashua

DOI10.1109/cvpr.2000.855871

Citations46

TL;DR

A new image sequence is created showing the scene from arbitrary viewing position and arbitrary time, made use of a newly discovered tool, the "dual Htensor" that connects together three views of a coplanar configuration of (unlabeled) static and moving points.

Abstract

We consider a scene, containing many objects moving with constant velocity along straight line paths, seen from three reference viewpoints at three different times. The scene may even consist only of moving objects with no static features. We wish to create a new image sequence showing the scene from arbitrary viewing position and arbitrary time. We make use of a newly discovered tool, the "dual Htensor" that connects together three views of a coplanar configuration of (unlabeled) static and moving points. The newly synthesized images use constant velocity in the world to achieve realistic and physically correct images.

Keywords

Computer Science

Plenoptic modeling

1,264 Citations1995Leonard McMillan, Gary Bishop

This work uses the "plenoptic function" of Adelson and Bergen to provide a concise problem statement for image-based rendering paradigms, such as morphing and view interpolation, and presents an image-based rendering system based on sampling, reconstructing, and resampling the plenoptic function.

Lecture notes in computer scienceHierarchical model-based motion estimation

1,155 Citations1992James R. Bergen, P. Anandan +2 more

This paper describes a hierarchical estimation framework for the computation of diverse representations of motion information that constrains the overall structure of the motion estimated, a local model that is used in the estimation process, and a coarse-fine refinement strategy.

Feature-based image metamorphosis

913 Citations1992Thaddeus Beier, Shawn Neely

View PDF

View morphing

725 Citations1996Steven M. Seitz, Charles R. Dyer

A simple extension to image morphing that correctly handles 3D projective camera and scene transformations is introduced that may be applied to photographs and drawings, as well as rendered scenes.

IEEE Computer Graphics and ApplicationsVideo mosaics for virtual environments

722 Citations1996Richard Szeliski

This article presents algorithms that align images and composite scenes of increasing complexity-beginning with simple planar scenes and progressing to panoramic scenes and, finally, to scenes with depth variation.

Panoramic mosaics by manifold projection

279 Citations2002Bezalel Peleg, Jessica Herman

Manifold projection enables the creation of panoramic mosaics from video sequences under more general conditions, and in particular the unrestricted motion of a hand-held camera.

IEEE Transactions on Visualization and Computer GraphicsNovel view synthesis by cascading trilinear tensors

151 Citations1998Shai Avidan, Amnon Shashua

This work presents a new method for synthesizing novel views of a 3D scene from two or three reference images in full correspondence through the use and manipulation of an algebraic entity, termed the "trilinear tensor", that links point correspondences across three images.

Lecture notes in computer scienceHomography Tensors: On Algebraic Entities That Represent Three Views of Static or Moving Planar Points

68 Citations2000Amnon Shashua, Lior Wolf

A 3 ×3 × 3 × 3 tensor Hijk and its dual Hijk which represent the 2D projective mapping of points across three projections (views) and it concatenates two homography matrices to represent the joint mapping across three views.

Lecture notes in computer scienceWhat can two images tell us about a third one?

68 Citations1994Olivier Faugeras, Luc Robert

View PDF

Interpolating view and scene motion by dynamic view morphing

60 Citations2003R.A. Manning, Charles R. Dyer

The problem of view interpolation for dynamic scenes is introduced, specifically concerned with interpolating between two reference views captured at different times, so that there is a missing interval of time between when the views were taken.