In this paper, we study the structure from motion problem as a constrained nonlinear least squares problem which minimizes the so called reprojection error subject to all constraints among multiple images. By converting this constrained optimization problem to an unconstrained one, we obtain a multiview version of the normalized epipolar constraint of two views. Such a multiview normalized epipolar constraint serves as a statistically optimal objective function for motion (and structure) estimation. Since such a function is defined naturally on a product of Stiefel manifolds, we show how to use geometric optimization techniques to minimize it. We present experimental results on real images to evaluate the proposed algorithm.
