ISPRS Commission III, Vol.34, Part 3A „Photogrammetric Computer Vision“, Graz, 2002 
  
ESTIMATION OF AND VIEW SYNTHESIS WITH THE TRIFOCAL TENSOR 
Helmut Mayer 
Institute for Photogrammetry and Cartography, Bundeswehr University Munich, D-85577 Neubiberg, Germany 
Helmut.Mayer@UniBw-Muenchen.de 
KEY WORDS: Visualization, Orientation, Hierarchical, Matching, Geometry 
ABSTRACT 
In this paper we propose a robust hierarchical approach for the estimation of the trifocal tensor. It makes use of pyramids, the sub-pixel 
Förstner point operator, least squares matching, RANSAC, and the Carlsson-Weinshall duality. We also show how the trifocal tensor can be 
utilized for an efficient view synthesis which we have optimized by parameterizing it according to the epipolar lines. 
1 INTRODUCTION 
In photogrammetry relative orientation and bundle block adjustment 
employing the collinearity equation are standard procedures. The 
reasons why additional orientation procedures based on projective 
geometry nevertheless are a viable alternative and partly are even 
necessary are: 
e These orientation procedures make direct solutions available, 
i.e., no approximate values and no iterations are needed. This 
is often essential for automatic procedures in variable geometry 
close range applications. 
e With these orientation procedures linear relations between mea- 
surements in the images arise without the need to reconstruct 
three dimensional (3D) geometry explicitly. This is especially 
important for applications such as video communication, where 
the goal is to generate artificial views and not 3D geometry. 
Representation and projection based on projective geometry are 
standard in computer graphics. 
Linear projective relations are known in photogrammetry for quite a 
while theoretically, though they have seldom been used in practice. 
An account of the history of linear methods in photogrammetry is, 
e.g., given in (Brandstátter, 1996). 
In this paper we concentrate on two things: The robust estimation 
of the trifocal tensor from three images and its utilization for view 
synthesis. Our major achievements are as follows: 
e By means of a hierarchical approach based on image pyramids 
we reduce the search space. Efficiency but also robustness are 
improved considerably. Highly precise conjugate points are ob- 
tained from a least-squares matching of points obtained from 
the Fórstner operator (Fórstner and Gülch, 1987) 
e We use the Carlsson-Weinshall duality to calculate a solution 
for the trifocal tensor from a minimum of six point triplets. This 
is the basis for a RANSAC (Fischler and Bolles, 1981) based 
robust algorithm. 
e We have optimized the view synthesis scheme proposed in 
(Avidan and Shashua, 1998) by linearly projecting the points 
as proposed in (Hartley and Zisserman, 2000) and, particularly, 
by parameterizing the points according to the epipolar lines. 
In Section 2 we introduce basic concepts and notations. They com- 
prise the fundamental matrix as it is used as a basic building block of 
our orientation procedure and the essential matrix as the view syn- 
thesis relies on calibration. Section 3 describes the trifocal tensor 
and in Section 4 the point transfer based on the trifocal tensor is de- 
tailed. In Section 5 we show how the trifocal tensor can be estimated 
from image data. Section 6 summarizes the algorithm we use to es- 
timate depth, i.e., disparity, from two images. This is the basis for 
view synthesis presented in Section 7. We end up with conclusions. 
2 BASICS OF LINEAR ORIENTATION 
2.1 Homogeneous Coordinates 
Homogeneous coordinates are derived from Euclidean coordinates 
by adding an additional coordinate and free scaling. Generally, for 
two dimensional (2D) and 3D points holds (Hartley and Zisserman, 
2000): 
In our notation we distinguish homogeneous 2D and 3D vectors x 
and X, respectively, as well as matrices P, which represent the same 
object also after a change of the scaling factor A (bold), from Eu- 
clidean vectors z and .X as well as matrices R (bold italics). 
Straight lines 1 in 2D and planes P in 3D can also be described by pa- 
rameters termed homogeneous coordinates. The incidence relation 
ax + by + c — 0 of point x and straight line 1, i.e., the point x lies on 
the straight line 1, reads with homogenous parameters for the straight 
linel = (abc)! x'1— I'x=x-1=0. 
2.2 Perspective Transformation 
In a local spatial coordinate system with origin in the camera (pro- 
jection) center O and with the image plane given by the equation 
z = c, the 3D object point P(X,Y, Z) is projected to the image 
point P' (z^, y) by z' 2 cX/Z t z, y' 2 cY/Z + yp. 
For the general case holds equation (1) given the point in object 
space X. The exterior orientation is described by projection center 
O(Xo) and rotation matrix R. The interior orientation is modeled by 
principal distance c, principal point (7, y; ), scale difference m of 
the coordinate axes and skew of the axes s. 
The 5 parameters chosen for the interior orientation are collected 
into the calibration matrix: 
c cs zh, 
K=| 0 c(1+m) yj, 
0 0 1 
A- 211