Prototypes of
Objects
Constraints
S
User input
Features
Camera Centered
| Constrainis -
Geometrical
Constraints
Viewpoint
Registration
Scene
Description ve 3—D model
World Centered
Figure 1: System overview
additional certainty map is created containing a reliability mea-
sure for the depth value. The certainty measurement is a com-
bination of NCC? between left and right image and the image
gradient (fig. 2 right).
In addition to the estimation of the depth maps, regions and con-
tours are extracted. The segmentation into regions uses the cri-
teria of the same orientation of surface points found in the depth
map. The details of the stereo pair processing and the segmenta-
tion can be found in [8][4].
The central module of the system is the interpretation. It assigns a
semantic meaning described in the knowledge base to the features
extracted in the image processing pipeline before. The knowl-
edge is formulated in a semantic net [6] and structured into three
layers of abstraction (fig. 3): The top layer or scene layer de-
scribes the world in terms that are highly symbolic. The middle
layer called world centered layer describes the appearance of the
objects found in the model world in 3-D space and in absolute
world coordinates. The bottom layer describes the objects ap-
pearance in camera centered coordinates, i.e. in 2-D space.
part-o part-of
is-a
con-of [ House | part-of
part-of part-of scd
Erie Wall Surface
Figure 3: Example semantic net (part)
The objects are represented as nodes in the net. The nodes are
connected via special edges or links: The part-of link enables
2 . .
^Normalized Cross Correlation
594
an object decomposition. The concrete-of (con-of) link connects
different layers of abstraction. The is-a link permits inheritance
of attributes from general to specialised nodes. Another link is
the instance-of link. It connects an instance node that was build
up during the interpretation with its prototype in the knowledge
base.
The creation of instances that describe the real scene is the goal
of the interpretation. It assigns those node types to objects that
are found in the scene, for example an instance of the node House
in fig. 3. The process of the interpretation is described in [6] in
detail. It is based on hypotheses and the their verification. In the
beginning of the interpretation a hypothesis House is established.
In the following the interpretation creates sub-hypotheses for ob-
ligate parts (like walls) of the house and tries to verify them in
the image. After all obligate hypotheses have been verified the
higher hypothesis House is validated.
From the resulting scene description geometrical constraints are
selected. These are together with the measured features from the
image processing pipeline the input of the surface reconstruction
module, which is described in the following sections.
3 Surface and Camera Representation
To achieve our goal to optimize the resulting 3-D model accord-
ing to some constraints, we need a scene representation that al-
lows parts to be moved around under the influence of an global
optimization algorithm.
As can be seen in figure 4, each part of the model (e.g. a wall
or a roof) is represented as a plane in space. The 3-D model
edges result from intersections between two neighbouring parts,
thus leading to a polygon description of each model element. To
control its position and orientation, each model part is assigned a
local coordinate system. The origin t of the system is positioned
above the center of the wall with the three axes u, v, w spanning
a righthanded coordinate system. The vector (1, 1, 1) has the op-
posite direction of the surface normal. A local coordinate system
for each part is necessary since the desired global optimization
is sensitive to inhomogeneous coordinates. With the proposed
local system all coordinates are treated with equal influence on
optimization.
The scene description consists of elements of the actual model
such as walls, as well as cameras. For camera representation the
cahv-model is used [7]. There are six degrees of freedom for
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B5. Vienna 1996
the extri
a model
surface
We prop
judge pr
tively gc
Every ct
C corres]
restrictic
part pos
function
Two diff
suremen
within o
perpendi
4.1 throı
Measure
like dept
4.5 and
integrate
4.1 An
Model p
them. Ir
ofthe m
constrair
function
where y
the actuz
volved p
of the co
