In: Paparoditis N., Pierrot-Deseilligny M.. Mallet C. Tournaire О. (Eds). I APRS. Vol. XXXV1I1. Pan ЗА - Saint-Mandé, France. September 1-3. 2010
OPTIMAL VANISHING POINT DETECTION AND ROTATION ESTIMATION OF
SINGLE IMAGES FROM A LEGOLAND SCENE
Wolfgang Forstner
University Bonn. Institute for Geodesy and Geoinformation
Department of Photogrammetry. Nussallee 15, 53121 Bonn, Germany
wf@ipb.uni-bonn.de. http://ipb.uni-bonn.de
Commission Ill/I
The paper presents a method for automatically and optimally de
termining the vanishing points of a single image, and in case the
interior orientation is given, the rotation of an image with respect
to the intrinsic coordinate system of a lego land scene. We per
form rigorous testing and estimation in order to be as independent
on control parameters as possible. This refers to (1) estimating
vanishing points from line segments and the rotation matrix. (2)
to testing during RANSAC and during boosting lines and (3) to
classifying the line segments w. r. t. their vanishing point. Spher
ically normalized homogeneous coordinates are used for line seg
ments and especially for vanishing points to allow for points at in
finity. We propose a minimal representation for the uncertainty of
homogeneous coordinates of 2D points and 2D lines and rotations
to avoid the use of singular covariance matrices of observed line
segments. This at the same time allows to estimate the parame
ters with a minimal representation. The vanishing point detection
method is experimentally validated on a set of 292 images.
1 INTRODUCTION
The orientation of a camera with respect to a world coordinate
system is a classical task of photogrammetry. It can only be de
termined in case some information about the w'orld is available
and can be identified in the image. Determining the camera pose
using reference points or lines visible in the image are classical
and use either the spatial resection or the direct linear transforma
tion. depending on whether the camera is calibrated or not. Re
ducing the amount of prior knowledge leads to the area of inverse
perspective, where partial information about the object and the
image can be derived using generic geometric properties of the
scene, especially the existence of parallel or orthogonal straight
lines. Going beyond, recent attempts to exploit illumination ef
fects visible in an image allow to derive rough estimates of the
orientation of the camera or the direction of the light source.
This paper presents a method for detecting vanishing points and
possibly the rotation matrix of the camera w. r. t. the intrinsic
coordinate system of a Lego land or Manhattan world with two
or three mutually orthogonal sets of straight lines visible in the
image. The paper is motivated by two observations of previous
approaches: (1) The difficulty to represent the space of solutions,
the unit sphere in two or three dimensions, within a search or es
timation process and (2) lack of a standard algorithmic sequence
for statistically rigorously inferring the desired parameters.
The main goal of the paper therefore is to apply rigorous testing,
classification and estimation in all steps of a stochastic algorithm
in order to reduce the number of control parameters and to obtain
optimal results in terms of reliability and accuracy. To exploit
the framework of projective geometry, w'here uncertain elements
regularly are characterized with singular covariance matrices and
during estimation require additional constraints, we propose a
minimal representation of the uncertainty and the parameters to
be estimated. These tw ; o means at the same time solve the two
above mentioned problems of representation and rigorous esti
mation.
The paper is organized as follows. We first give a short review
on previous approaches for handling the two mentioned prob
lems within vanishing point detection and rotation estimation.
We then introduce a minimal representation overcoming singu
larities, which is used for a method for vanishing point detection
and rotation estimation. We experimentally validate the method
on a set of 292 images.
Notation. We distinguish between the name, say of an entity
and its representation, say x. We distinguish homogeneous vec
tors or matrices, say the 3-vector x or the 3 x 3-matrix K. from
Euclidean vectors or matrices, say the 2-vector x and the 3x3-
matrix R. Stochastical variables, say x, are underscored. The unit
matrix is denoted with l n . the skew symmetric 3 x 3-matrix S(x)
of a 3-vector induces the cross product, thus x x y = S(x)y.
In case an equation contains homogeneous entities on both sides,
the equality sign = means equality up to a factor ^ 0. Concate
nation of scalars, vectors and matrices follows Matlab style:
horizontal concatenation reads [a, b], vertical reads [a; b\.
2 RELATED WORK
Vanishing point detection has been attacked at least since Barnard's
influential paper 1983, addressing the unit sphere representation
of the space of all vanishing points. When referring to one vanish
ing point all lines not going through this point are outliers highly
robust methods are required, e. g. clustering [Straforini et al.,
1992]. the Hough transform [Tuytelaars et al., 1998] and random
sample consensus [Wildenauer and Vincze, 2007]. Heuvel ] 1998]
included an estimation of the rotation matrix. A good review is
given by Rother [2000]. However, quite a number of recent pub
lications address special aspects, such as including lens distor
tion [Grammatikopoulos et al., 2007], focussing on road scenes
[Kong et al., 2009]. efficient clustering processes [Schmitt and
Priese. 2009]. or a new method based on the socalled J-linkage
algorithm for efficient clustering [Tardif. 2009].
Statistical modelling in the context of vanishing point detection
has been adressed by Collins and Weiss 11990] w-ho used models
for the uncertainty of lines and vanishing point for optimal esti
mation, [Heuvel. 1998] who tracked the uncertainty through the
sequence of decision steps or Coughlan and Yuille [2003], who
include generic knowledge of the distribution of the lines, which
was the basis for Deutscher et al. [2002]. where also the principle
distance was estimated, and Schindler and Dellaeit [2004], who
use an expectation-maximization approach to find more that one
triplet of vanishing points.
Our approach, similar to the one of Heuvel [1998]. tracks the
uncertainty from the automatic line detection, via the vanishing
157
