Novel view synthesis in tensor space
Published 22 November 2002
Shai Avidan, Amnon Shashua
Citations191
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work presents a new method for synthesizing novel views of a 3D scene from few model images in full correspondence by derivation of a tensorial operator that describes the transformation from a given tensor of three views to a novel Tensor of a new configuration of threeViews.
Abstract
We present a new method for synthesizing novel views of a 3D scene from few model images in full correspondence. The core of this work is the derivation of a tensorial operator that describes the transformation from a given tensor of three views to a novel tensor of a new configuration of three views. By repeated application of the operator on a seed tensor with a sequence of desired virtual camera positions we obtain a chain of warping functions (tensors) from the set of model images to create the desired virtual views.
Keywords
Computer Science
Light field rendering
3,737 Citations1996Marc Levoy, Pat Hanrahan
A simple and robust method for generating new views from arbitrary camera positions without depth information or feature matching, simply by combining and resampling the available images.
The MIT Press eBooksThe Plenoptic Function and the Elements of Early Vision
1,479 Citations1991Edward H. Adelson, James R. Bergen
Plenoptic modeling
1,264 Citations1995Leonard McMillan, Gary Bishop
This work uses the "plenoptic function" of Adelson and Bergen to provide a concise problem statement for image-based rendering paradigms, such as morphing and view interpolation, and presents an image-based rendering system based on sampling, reconstructing, and resampling the plenoptic function.
Lecture notes in computer scienceHierarchical model-based motion estimation
1,155 Citations1992James R. Bergen, P. Anandan +2 more
This paper describes a hierarchical estimation framework for the computation of diverse representations of motion information that constrains the overall structure of the motion estimated, a local model that is used in the estimation process, and a coarse-fine refinement strategy.
QuickTime VR
1,145 Citations1995Shenchang Eric Chen
This paper presents a new approach which uses 360-degree cylindrical panoramic images to compose a virtual environment which includes viewing of an object from different directions and hit-testing through orientation-independent hot spots.
View interpolation for image synthesis
1,106 Citations1993Shenchang Eric Chen, Lance R. Williams
The MIT Press eBooksComputational Models of Visual Processing
1,080 Citations1991
Part 1 The task of vision: the plenoptic function and the elements of early vision, Edward H.Adelson and James R.Bergen and Michael S.Landy.
Lecture notes in computer scienceWhat can be seen in three dimensions with an uncalibrated stereo rig?
833 Citations1992Olivier Faugeras
This paper addresses the problem of determining the kind of three- dimensional reconstructions that can be obtained from a binocular stereo rig for which no three-dimensional metric calibration data is available, and shows that even in this case some very rich non-metric reconstructions of the environment can nonetheless be obtained.
Panoramic mosaics by manifold projection
279 Citations2002Bezalel Peleg, Jessica Herman
Manifold projection enables the creation of panoramic mosaics from video sequences under more general conditions, and in particular the unrestricted motion of a hand-held camera.
IEEE Transactions on Pattern Analysis and Machine IntelligenceAlgebraic functions for recognition
265 Citations1995Amnon Shashua
The trilinearity result is shown to be of much practical use in visual recognition by alignment-yielding a direct reprojection method that cuts through the computations of camera transformation, scene structure and epipolar geometry.
On the geometry and algebra of the point and line correspondences between N images
223 Citations2002Olivier Faugeras, Bernard Mourrain
The formalism of the Grassmann-Cayley algebra is proposed to use as the simplest way to make both geometric and algebraic statements in a very synthetic and effective way (i.e. allowing actual computation if needed).
International Journal of Computer VisionStructure from motion using line correspondences
208 Citations1990Minas E. Spetsakis, John Aloimonos
A theory is presented that describes a closed form solution to the motion and structure determination problem from line correspondences in three views, compared with previous ones that are based on nonlinear equations and iterative methods.
IEEE Computer Graphics and ApplicationsCreating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter
207 Citations1986Ned Greene, Paul S. Heckbert
This application presents a method for creating Omnimax projections from multiple perspective views, and uses a space-variant filter for this application that computes a weighted average over an arbitrarily oriented elliptical area.
Matching constraints and the joint image
193 Citations2002Bill Triggs
The geometry of multi image perspective projection and the matching constraints that this induces on image measurements are studied and their complex algebraic interdependency is captured by quadratic structural simplicity constraints on the Grassmannian.
DSpace@MIT (Massachusetts Institute of Technology)Example Based Image Analysis and Synthesis
172 Citations1993David Beymer, Amnon Shashua +1 more
The mapping from novel images to a vector of ``pose'''' and ``expression'''' parameters can be learned from a small set of example images using a function approximation technique that it analysis network.
3-D scene representation as a collection of images
162 Citations2002Stéphane Laveau, Olivier Faugeras
This work addresses the problem of the prediction of new views of a given scene from existing weakly or fully calibrated views called reference views, and derives an algorithm from the constraints between the images to predict the new views.
A linear method for reconstruction from lines and points
162 Citations2002Richard Hartley
The trifocal tensor is shown to be essentially identical to a set of coefficients introduced by Shashua (1994) to effect point transfer in the three-view case and to be extended to allow for the computation of the trifoc tensor given any mixture of sufficiently many line and point correspondences.
Physically-valid view synthesis by image interpolation
144 Citations2002Steven M. Seitz, Charles R. Dyer
It is shown that two basis views are sufficient to predict the appearance of the scene within a specific range of new viewpoints and that generating this range of views is a theoretically well-posed problem, requiring neither knowledge of camera positions nor 3D scene reconstruction.
Trilinearity of three perspective views and its associated tensor
133 Citations2002Amnon Shashua, Michael Werman
It is shown that a permutation of the the trilinear coefficients produces three homography matrices (projective transformations of planes) of three distinct intrinsic planes, respectively, which yields the result that 3D invariants are recovered directly-simply by appropriate arrangement of the tensor's coefficients.
Relative affine structure: theory and application to 3D reconstruction from perspective views
113 Citations1994Shashua, Navab
An affine framework for perspective views is proposed, captured by a single extremely simple equation based on a viewer-centered invariant the authors call relative affine structure, that unifies previous work-including Euclidean, projective and affine-in a natural and simple way.
Direct methods for visual scene reconstruction
101 Citations2002Richard Szeliski, Sing Bing Kang
This work presents some of the recent work in this area, which is based on the registration of multiple images (views) in a projective framework, which directly solve a least-squares estimation problem in the unknown structure and motion parameters, which leads to statistically optimal estimates.
Representation of scenes from collections of images
95 Citations2002Raman Kumar, P. Anandan +3 more
The central thesis of this paper is that the traditional approach to representation of information about scenes by relating each image to an abstract three dimensional coordinate system may not always be appropriate.
Rendering real-world objects using view interpolation
85 Citations2002Tomáš Werner, Roger D. Hersch +1 more
This work presents a new approach to rendering arbitrary views of real-world 3D objects of complex shapes, and shows that this combination can be linear, assuming proximity of the views, and suggests how the visibility of constructed points can be determined.
Lecture notes in computer scienceThe rank 4 constraint in multiple (≥3) view geometry
77 Citations1996Amnon Shashua, Shai Avidan
First general results on any number of views of trilinear tensors across m>3 views are shown, and given two views, all the induced homography matrices are embedded in a four-dimensional linear subspace.
Lecture notes in computer scienceWhat can two images tell us about a third one?
68 Citations1994Olivier Faugeras, Luc Robert
Reconstruction from image sequences by means of relative depths
59 Citations2002Anders Heyden
A Novel Approach to Graphics
35 Citations1992Tomaso Poggio, Roberto Brunelli
This work shows that it can optimally represent the set of 2D images produced by the point features of a rigid 3D model as two lines in two high-dimensional spaces, and decribes a working recognition system in which these spaces are represented discretely in a hash table.
DSpace@MIT (Massachusetts Institute of Technology)Why Stereo Vision is Not Always About 3D Reconstruction
23 Citations1993W. Ericl . Grimson
It is argued that for tasks like recognition, figure/ ground separation is more important than 3D depth reconstruction, and a stereo algorithm is demonstrated that supports figure/ground separation without 3D reconstruction.
Robust recovery of camera rotation from three frames
20 Citations1996Benny Rousso, Shai Avidan +2 more
A robust approach for recovering camera rotation is presented, which does not assume any specific scene structure (e.g. no planar surface is required), and which avoids prior computation of the epipole.
Lecture notes in computer scienceThe quadric reference surface: Applications in registering views of complex 3D objects
11 Citations1994Amnon Shashua, Sebastian Toelg
A nominal quadratic transformation is proposed that can be used in practice to facilitate the process of achieving full point-to-point correspondence between two grey-level images of the same (arbitrary) object.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Robust algebraic invariant methods with applications in geometry and imaging</title>
8 Citations1995Eamon B. Barrett, Paul M. Payton +1 more
Non-standard methods of deriving algebraic invariants are introduced and two types of applications of these invariant algorithms are demonstrated, both of which are extensible from the linear fractional central projection camera model to weak perspective and certain non-central projection camera models.
DSpace@MIT (Massachusetts Institute of Technology)Image-Based View Synthesis
4 Citations1997Shai Avidan, Theodoros Evgeniou +2 more