photogram-
ınd machine
precision of
ves for Pho-
(3), pp.176-
ereo vision,
tern Recog-
' Vision, pp.
igital stereo
70.
e matching
lications of
Lausanne.
290, Recog-
n image, In-
n, 5(2), pp-
tereo corre-
ceedings of
4-241.
object rec-
nages", Ar-
omatic reg-
hine Vision
ment-based
aphics and
ptual orga-
E Transac-
itelligence,
tection and
erial imag-
ing Work-
4, Feature
n multiple
Vision and
Recovering Unknown Focal Lengths in Self-Calibration:
An Essentially Linear Algorithm and Degenerate Configurations
G.N. Newsam D.Q. Huynh M.J. Brooks H.-P. Pan
{gnewsam,du,mjb,heping} @cssip.edu.au
Centre for Sensor Signal and Information Processing
Signal Processing Research Institute
Technology Park, The Levels, SA 5095, Australia
KEY WORDS: Stereo vision, degeneracy, epipolar equation, fundamental matrix, self-calibration, relative orientation.
ABSTRACT
If sufficiently many pairs of corresponding points in a stereo image pair are available to construct the associated fundamental
matrix, then it has been shown that 5 relative orientation parameters and 2 focal lengths can be recovered from this fundamental
matrix. This paper presents a new and essentially linear algorithm for recovering focal lengths. Moreover the derivation of the
algorithm also provides a complete characterisation of all degenerate configurations in which focal lengths cannot be uniquely
recovered. There are two classes of degenerate configurations: either one of the optical axes of the cameras lies in the plane
spanned by the baseline and the other optical axis; or one optical axis lies in the plane spanned by the baseline and the vector
that is orthogonal to both the baseline and the other axis. The result that the first class of configurations (i.e. ones in which
the optical axes are coplanar) is degenerate is of some practical importance since it shows that self-calibration of unknown
focal lengths is not possible in certain stereo heads, a configuration widely used for binocular vision systems in robotics.
1 INTRODUCTION
Relative orientation is the problem of recovering the parame-
ters defining the rotation and translation direction relating
two calibrated camera views from a set of corresponding
image points. This has long been studied by photogram-
metrists [4, 16], and more recently by the computer vision
community [3, 7, 9, 17]. 5 parameters suffice to define the
relative orientation of the two cameras, 3 describing the ro-
tation and 2 the direction of translation. An essentially linear
algorithm for their recovery was proposed by Stefanovic [15]
and revived by Longuet-Higgins [9], based on the coplanarity
constraint between corresponding points. This involves com-
puting the 3 x 3 essential matrix [3] associated with the stereo
pair, either by solving a linear system of equations derived
from eight or more pairs of corresponding points [9], or by
finding the vector associated with the smallest singular value
of the associated system [3, 17]. Once the essential matrix is
available, it can be decomposed into a product of a rotation
matrix and a skew symmetric matrix derived from the base-
line (see [3] for a comprehensive discussion of algorithms for
this).
Given these solutions, the question of how to deal with un-
calibrated cameras has come to the fore in the last few years.
Here, the problem is to simultaneously calibrate the cameras
and recover the viewing geometry from a given set of images,
i.e. to recover both the intrinsic (interior orientation) and
extrinsic (relative orientation) parameters. We shall refer to
this process as self-calibration. Faugeras et al. [2] developed
a model for the interior orientation of a general uncalibrated
pinhole camera in terms of 5 intrinsic parameters. Moreover
a generalisation of the essential matrix, the fundamental ma-
trix [10, 11], can be defined and derived from the data that
is a function of both the intrinsic and extrinsic parameters.
Even if the same camera is used to take both images in a
stereo pair, however, these cannot be recovered simultane-
ously along with the 5 extrinsic parameters from the funda-
mental matrix as this matrix is determined by only 7 inde-
pendent parameters in total. If three or more images of the
same scene taken by the same camera are available though,
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B3. Vienna 1996
then the 10 orientation parameters can be recovered from the
set of fundamental matrices derived from all possible pairs of
images [2]. More recently Niini [12] has presented a promis-
ing new approach for recovering these parameters based upon
setting up linear systems for certain intermediate quantities.
An interesting special case in which self-calibration is possible
from a single pair of images attains when the cameras tak-
ing the images are calibrated up to unknown focal lengths.
In this case there are only 2 unknown intrinsic parameters
(the unknown focal length in each image) in addition to the
5 extrinsic parameters. Since the fundamental matrix has 7
independent parameters there would seem to be sufficient in-
formation to carry out self-calibration. Hartley [6] has shown
that this is indeed possible through an approach based on the
singular value decomposition (SVD) of the fundamental ma-
trix. Pan et al. [13] presented an alternative closed-form ex-
pression for the focal lengths as roots of certain cubics, along
with an iterative least-squares technique for refining parame-
ter estimates [14]. Recently, however, Huynh et al. [8] have
shown that both algorithms fail for certain degenerate cam-
era configurations: in particular, when the optical axes of the
cameras are coplanar the problem becomes degenerate and
unique focal lengths for each camera cannot be determined.
This has some practical importance since in the stereo heads
used in robotic vision it is common for camera motion to be
restricted to vergence only, so that the optical axes and the
baseline between them are confined to the horizontal plane.
The purpose of this paper is to present a new and simpler
algorithm for self-calibration of focal lengths through the so-
lution of a linear system of equations. The approach is sim-
ilar in spirit to that of Niini [12] and rests very much on a
linear algebraic formulation of the problem, but it is based
on a different representation than that used in [12]. It fol-
lows Hartley in using the SVD, but is considerably simpler in
that it does not require solution of any further higher order
systems. Moreover a by-product of the approach is a com-
plete characterisation of degenerate configurations in which
self-calibration is not possible: these are identified as those
configurations which make the linear system singular. In par-
575
