nbul 2004
> general
'onitoring
ymatic ID
> analysis
' used to
cation of
cenes are
>sent and
mation to
Azuma et
curacy or
nbination
tracking
GPS and
e existing
lay of a
ipplied to
of fixed
tion. The
is one of
2000). It
lings, so
important
specific
rientation
niques is
al., 1996)
extracted
n from a
ed reality
ors of a
lected by
indmarks
ronment.
ment an
ining the
idmarks).
d exists —
; various
d objects
integrate
support
' devices
utational
ces and
example
useful to
ronment.
nt exists
ng them.
pjects as
igmented
s to be
; leads to
and the
proaches
:quential
nage the
d images
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B3. Istanbul 2004
by image-to-image matching (e.g. based on straight line
features) to get the accurate position and orientation for the real
world images taken by the camera later. We do not use such
comparison methods but a geometric 3D model of the
environment, providing also 3D information about objects.
pm... 4
Figure 2. 3D city model (centre of the city of Stuttgart)
The basic information to provide orientation and navigation by
image analysis is the availability of a suitable geometric “world
model”. As example a (geo-referenced) 3D city model
represents such a world model, see Figure 2. Nowadays 3D city
models for large areas are easy to provide, as in the meantime
automatic methods for generation of such models are available
(Brenner, 2000; Wolf, 1999). For example the City Surveying
Office of Stuttgart offers such a 3D model of the centre of the
city for sale.
2.2.1 Feature extraction: As the principle of our method is
to co-register image and model data using prominent features in
the real world and the world model, suitable features must be
identified in the image, by image analysis techniques. The main
objects of interest within an urban environment are human made
objects, e.g. buildings. If we have a closer look to these objects
so the prominent features of them are texture and edges. The
edges of human made objects often can be approximated by
straight lines, therefore we decided to select edges as prominent
features. Also objects in the 3D city model are modelled by
straight lines.
For edge extraction and straight line approximation different
operators are available, e.g. Burns-Algorithm (Burns, 1986), but
also the Hough-Transform (Hough, 1962) is a suitable
algorithm to extract straight lines. Using these algorithms we
can prepare the collected data (images) and extract the
necessary feature information. In Figure 3 the result of these
process is displayed and the extracted straight lines are marked
in red.
Ji Nebel 9 RE 2 any aJ Vies
B ORTI DU pee
ES EE
— SO
AS |
a
RT S
mm
Figure 3. Feature extraction and line detection
905
On the one hand side the base information for our approach is
provided by the straight line extraction and on the other hand
side by the world model (73D city model). The problem that
occurs now is to select candidate objects within the world
model for further processing. Additional information provided
by simultaneous collection of image data as well as rough
orientation and position information can help to identify one or
a set of candidate objects. By using our prototype for direct
collection of exterior orientation parameters we are able to solve
this problem. The prototype integrates an image sensor (CCD
camera), an orientation module for goniometry and a GPS
receiver, which provides rough position information.
azimuth
angle
vertex
angle
CCD »€
camera
Figure 4. Prototype sensor
To extract required data for further processing we have to query
the model for objects, which are contained in the viewing
frustum of the camera and select the visible objects (see Fig. 5).
The data collected by the prototype sensory can help us to
extract these model data by an integrated processing of sensor
and model data.
.&,
"A.
Figure 5. Exttaction of visible objects
2.2.2
World Model) to image registration problem the features of the
selected and extracted 3D object model and the features
extracted in the 2D image must be assigned. This is the task of
determining straight lines in image and object space that are
belonging together. Automatic registration of 3D models to
Feature Matching: To solve the model (Augmented
images is a fundamental and open problem. It is difficult
because it comprises two coupled problems: the correspondence
and the pose problem. Each sub problem is easy to solve only if
the other has been solved first. A classic approach for solving
these coupled problem is the hypothesize and test approach.
First a small set of image feature to model feature
correspondences are hypothesized, e.g. using sensor data
collected by the prototype sensor to minimize the search space.
Based on the result the object's pose is computed and the model
is back projected into the image. The pose is accepted if the
original and back projected image are sufficiently similar,
otherwise a new hypothesis is formed and the process is
repeated. As example the method of Beveridge and Riseman
(Beveridge&Riseman, 1995) uses a random start local search
with a hybrid pose estimation algorithm employing both full-
perspective and weak-perspective camera models. To simplify
the equations often linear affine approximations (weak-
