EXTRACTING SPATIAL INFORMATION FROM DIGITAL 
VIDEO IMAGES USING MULTIPLE STEREO FRAMES 
Martin Braess, Geodátisches Institut der RWTH Aachen 
Commission 11/5 
KEY WORDS: Acquisition, Automation, Stereoscopic, Three-dimensional, Close_Range, Scene Reconstruc- 
tion, Feature Based Matching. 
ABSTRACT: 
This paper describes a feature-based approach for the reconstruction of 3D scene geometry. Digital images 
are taken from a moving surveying vehicle. Straight lines are matched. We minimize a cost function that 
incorporates feature attributes and relations between features using a branch-and-bound algorithm. An example 
for the line matching in a short sequence with two image pairs is presented. 
1 Introduction 
1.1 Problem Description 
A surveying vehicle that collects various data for GIS 
databases is given. The vehicle is equipped with a 
GPS receiver and wheel sensors to determine its abso- 
lute position. The wheel sensors furnish 2500 impulses 
per turn which are registered with a hardware coun- 
ter. The counter values and GPS data are recorded 
once per second. 
In addition, the vehicle contains a digital video ca- 
mera pair. We use two PULNiX TM 9700 cameras. 
They synchronously capture grey scale images with 
standard video resolution on a PC. The cameras are 
mounted on a stable aluminium profile, thus their re- 
lative orientation remains constant. Whenever an ob- 
ject of interest appears, an operator records one or 
more stereo pairs on a fixed disk. The counter values 
are stored with every image pair. 
The images have to be evaluated in postprocessing 
by an operator who is mainly interested in the position 
of distinct points and lines in 3D-space. Most objects 
of interest in the traffic environment are man-made 
and contain straight lines. The object reconstruction 
is possible if the correspondence problem is solved. In 
this paper we want to support the operator by sug- 
gesting correctly matched objects or features. 
If a feature is captured in two images, its position 
can be determined. In our approach, however, an ob- 
ject is followed in the image sequence in order to get 
both a higher accuracy and a higher reliability of the 
object position. Here, we confine ourselves to the use 
of interest points and straight lines only to describe 
objects in space. 
The outline of this paper is as follows: first, the ca- 
mera calibration and feature extraction is briefly de- 
scribed. The matching process consists of three sta- 
26 
ges: initial line matching, orientation of the camera 
pair and final matching. The paper is concluded with 
a practical example and a short discussion. 
1.2 Algorithm Outline 
The matching problem is often described as depending 
on three items: 
e feature attributes 
e relations between features 
e geometric constraints 
Feature attributes are used to determine the similarity 
of features to be matched. Typical attributes are grey 
values, line parameters or operator values. 
Relations between features, however, tell us whether 
a match is consistent or not: if two features in one 
image have a certain relation, the corresponding fea- 
tures in another image are likely to have the same 
relation. Typical relations are angles and distances 
between features. In some applications the assign- 
ment is based on relations only, this is the case of 
structural matching [8]. In principle, ternary or even 
higher order relations can be used but in this paper 
it is assumed that binary relations provide a sufficient 
describtion [2]. 
When the camera orientation is known, this infor- 
mation can be introduced in form of geometric con- 
straints. In our case, the relative orientation of the 
camera pair is known. Then we can impose the co- 
planarity constraint for points using two images: two 
points can only be matched if they are coplanar. 
Lines in space are completely described with 4 pa- 
rameters, lines in the image are described with 2 pa- 
rameters [7]. That is why any two lines in two images 
furnish exactly one line in space, there is no redun- 
dancy (the degenerated case where the 3D-line lies 
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B2. Vienna 1996 
in th 
ted 
'The 
the 1 
Th 
cisiol 
in ge 
form 
purp 
mini 
Th 
tiona 
check 
Ther 
consi 
proc 
2 
Befoi 
pair 
test 1 
red. 
toma 
Ther 
tion 
use t 
tions 
6 pai 
and . 
deled 
the r 
is m 
linea 
dr( 
‘Tests 
entat 
stem 
3 
The 
luati 
from 
ted 1 
point 
with 
Sti 
edge: 
then 
lengt 
Di 
point