ISPRS Commission II, Vol.34, Part 3A „Photogrammetric Computer Vision", Graz, 2002
a
b
c
Figure 5: Result I: Image triplet with points and epipolar lines. a) Epipolar lines for b) and c) b) and c) Epipolar lines for a) only
where aj is the first part A of the calibrated projection matrix of the
second camera.
The actual projection is done based on the optimized scheme pro-
posed in Section 4. The synthesized image is produced indirectly by
mapping the pixels via affine transformation obtained by the known
coordinates of triangle meshes in the given and the synthesized im-
age. Results are shown in Figures 7, 8, and 9. In all cases only the
translation vectors have been modified. The synthesized views give
an impression of the depth in the image. Compared to approaches
for view interpolation also the quality for viewing directions which
are not in the direction of the base vector T is reasonable (cf., e.g.,
Figure 9 for which the epipolar lines are nearly vertical). On the
other hand, one can see the problems arising from the height data.
Besides the fact that parts had too bad texture to be matched (black
holes in the depth map), also the boundary of structures with large
disparities such as the chair in Figure 8 are not well delineated.
8 CONCLUSIONS
In this paper we have presented means for the estimation of the trifo-
cal tensor as well as its use for view synthesis. All results presented
have been obtained totally automatically, without any user interac-
tion. The same parameters have been used for all examples. While
the estimation of the trifocal tensor based on pyramids, least squares
matching, and RANSAC works reliable for a wide range of imagery,
the end-to-end automation of view synthesis still is an intricate prob-
lem. There are two things we still have to cope with in-depth: Cam-
era calibration and depth estimation.
For camera calibration we have begun to implement the approach
presented in (Pollefeys and Van Gool, 1999, Hartley and Zisserman,
2000). The more complicated problem is depth estimation. Op-
posed to the determination of the orientation which is defined by
very few parameters and is therefore a highly redundant problem,
depth estimation aims at determining many parameters. Although
A - 216
b
Figure 6: Result II: Image triplet with points and epipolar lines. a) Epipolar lines for b) and c) b) and c) Epipolar lines for a) only
c
the approach we are using is relatively sophisticated, the results are
in many instances unstable and not really good. One way for im-
provement would be to use more images. This makes it computa-
tionally much more expensive as the simple epipolar geometry can-
not be used any more. Other ways for improvement were recently
shown in (Scharstein and Szeliski, 2002). As the most important
problem is the determination of approximate values, a combination
with direct sensors with possibly a lower resolution such as cheap
laser-scanners planned, e.g., for airbag inflation control, might be
considered for the application domain of video communication.
ACKNOWLEDGMENTS
The images used in Figures 4, 6, 8, and 9 are courtesy of the Robotvis
group at INRIA and the images utilized in Figures 5 and 7 stem from
ISPRS Working Group V/2.
We thank Peter Krzystek for making us available his code for least
squares matching.
REFERENCES
Avidan, S. and Shashua, A., 1998. Novel View Synthesis by Cas-
cading Trilinear Tensors. IEEE Transactions on Visualization and
Computer Graphics 4(4), pp. 293-306.
Brandstitter, G., 1996. Fundamentals of Algebro-Projective Pho-
togrammetry. In: Sitzungsberichte, Mathematisch-naturwissen-
schaftliche Klasse Abt. II, Mathematische, Physikalische und
Technische Wissenschaften, Österreichische Akdademie der Wis-
senschaften, Vol. II (1996) 205, pp. 57-109.
Carlsson, S., 1995. Duality of Reconstruction and Positioning from
Projective Views. In: IEEE Workshop on Representation of Visual
Scenes, Boston, USA.
Pc
ar
