AUTOMATED APPEARANCE-BASED BUILDING DETECTION IN TERRESTRIAL
IMAGES
Jan Böhm”, Norbert Haala, Peter Kapusy
Institute for Photogrammetry (ifp), University of Stuttgart, Germany
Geschwister-Scholl-Strasse 24D, D-70174 Stuttgart
Jan.Boehm@ifp.uni-stuttgart.de
KEY WORDS: Object Recognition, CAD, Orientation, Navigation, Augmented Reality
ABSTRACT:
We present a method for automated appearance-based detection of buildings in terrestrial images. The problem is stated as follows:
From an image with a given approximated exterior orientation and a three-dimensional CAD Model of the building, detect the exact
location of the building in the image. The method we have developed uses the combination of an imaging device and hardware to
approximately measure the exterior orientation. This paper presents a combination of our work on close-range photogrammetry and
virtual city models.
1. INTRODUCTION
With the increased availability of low-cost and low-power
consumption imaging devices we see a strong increase in their
integration into mobile devices such as laptops, personal digital
assistants (PDAs), mobile phones and so on. The combination
of mobile computational capabilities, imaging capabilities,
positioning capabilities and network access opens the door for a
variety of novel applications, such as pedestrian navigation
aids, mobile information systems and others usually referred to
as 'location-based services’. With these devices at hand our
interest is to exploit the capabilities of the imaging device for
photogrammetric processing.
Personal navigation and the provision of location dependent
information are key features of location aware applications. One
option to reach this goal is the application of augmented reality
(AR) techniques. These techniques are based on the overlay of
computer-generated graphics to the user’s actual view. The
computer graphics are generated based on a spatial model of the
visible environment. Of course the virtual computer graphic
objects have to be overlaid to their corresponding primitives in
the real world as the user observes them. For this reason the
accurate determination of the actual position and orientation of
the user is required in order to enable a precise mapping of the
data. This paper presents a method for the precise detection and
localization of buildings in terrestrial images and thereby also
provides the means for better navigation of users.
Within an urban environment AR can for example be applied
for the presentation of name labels or additional alphanumeric
data appearing to be attached to a side of a building. In addition
to the visualization of these virtual signposts, more specialized
applications could aim at the display of information based on
“X-ray vision” in order to present features normally not visible
for the user. Typical objects of interest are features hidden
behind the facades of a building like the location of rooms or
information on infrastructure like the position of power-lines.
The integration of augmented reality into a tourist information
System is another application for this kind of technique. The so-
called telepointing capability is an important feature in
* Corresponding author
implementing a intuitive user interface for location based
services (Fritsch et al. 2000).
In order to detect objects usually a model of the object has to be
available. In our case we need a model of the buildings to be
detected. Therefore one of the key components of our approach
is a 3D city model. The development of tools for the efficient
collection of 3D city models has been a topic of intense
research for the past years. In addition to Digital Height Models
and data representing streets and urban vegetation, building
models are the most important part thereof. Meanwhile a
number of algorithms based on 3D measurement from aerial
stereo imagery or airborne laser scanner data are available for
automatic and semi-automatic collection of 3D building
models. A good overview on the current state-of-the-art of
experimental systems and commercial software packages is
given in (Baltsavias et al. 2001).
Within this article we present a method to analyze the image of
an urban scene and reliably and robustly detect buildings. The
method uses the combination of an imaging device and
hardware to approximately measure the exterior orientation.
The result of the detection can be used both for AR applications
and for navigation purposes. Section 2 discusses the approach to
the problem of object recognition we have chosen. It contains
the description of the model representation, the feature
extraction and the recognition algorithm. The utilization of the
object recognition for the determination of the orientation
parameters using spatial resection is detailed in section 3. In
Section 4 we present the results of some typical test cases and
demonstrate the accuracy of our approach.
2. OBJECT RECOGNITION
When the task is to detect a three-dimensional shape in an
image, two general strategies for object representation are
available. One is the three dimensional representation of the
object which leads to a 3D to 2D matching problem, the other is
a two-dimensional representation which leads to a 2D to 2D
matching problem. While the former is the more general and
theoretically more appealing approach, there are several
practical problems, which often prevent its use. One of the
—491-
