Three-View Multibody Structure from Motion

A geometric approach to 3D motion segmentation from point correspondences in three perspective views and derives a rank constraint on the embedded correspondences from which one can estimate the number of independent motions, as well as linearly solve for the multibody trifocal tensor.

Abstract

We propose a geometric approach to 3-D motion segmentation from point correspondences in three perspective views. We demonstrate that after applying a polynomial embedding to the point correspondences they become related by the socalled multibody trilinear constraint and its associated multibody trifocal tensor, which are natural generalizations of the trilinear constraint and the trifocal tensor to multiple motions. We derive a rank constraint on the embedded correspondences, from which one can estimate the number of independent motions as well as linearly solve for the multibody trifocal tensor. We then show how to compute the epipolar lines associated with each image point from the common root of a set of univariate polynomials and the epipoles by solving a pair of plane clustering problems using Generalized PCA (GPCA). The individual trifocal tensors are then obtained from the second order derivatives of the multibody trilinear constraint. Given epipolar lines and epipoles, or trifocal tensors, one can immediately obtain an initial clustering of the correspondences. We use this clustering to initialize an iterative algorithm that alternates between the computation of the trifocal tensors and the segmentation of the correspondences. We test our algorithm on various synthetic and real scenes, and compare with other algebraic and iterative algorithms.