mera, 
era) 
  
  
ion 
each 
trian- 
depth 
con- 
angu- 
proxi- 
ithout 
patch 
xture 
iginal 
phics 
e 3D 
face 
e ge- 
parity 
1e left 
oma- 
ribed 
IMAGE SEQUENCE PROCESSING 
The tasks performed so far were straight forward ste- 
reoscopic image analysis. From the stereoscopic 
image pair a 3D surface approximation was extracted 
from a single camera view point together with a quali- 
ty measure of the estimated surface position. When 
complex scenes with occluding objects are to be ana- 
lyzed orwhenthe measurement quality of the surface 
geometry has to be improved the camera must be 
moved throughout the scene and the measurements 
from multiple view points have to be integrated into 
the 3D surface model. Therefore it is necessary to 
estimate the 3D motion of the camera and possible 
object motions in the scene fromthe image sequence 
and to fuse the multiple depth measurements into a 
consistent 3D scene model. 
mu nmn musinc-anatssisibusvmResi 
In this section an algorithm to directly estimate 3D 
scene motion from a monocular image sequence is 
described. For 3D motion estimation the object shape 
is assumed to be known. An initial estimate of the 
scene shape was generated from stereoscopic image 
analysis. When the initial estimate fails this depen- 
dency may affectthe analysis and will sometimes lead 
to estimation errors. As long as the initial shape ap- 
proximation is reliable, however, this dependency can 
be neglected. 
An object 
is defined as a rigid 3D-surface in space that is 
spanned by a set of N control points. A set of six 
motion parameters is associated with each object. 
Object motion is defined as rotation of the object con- 
trol points around the object center followed by a 
translation of the object center, measured between 
two successive image frames k and k+1. The object 
center G is the mean position vector of all N object 
control points. Each object control point Pj) at frame 
kis transformed to its new position Pj(x+1) in frame k+1 
according to the general motion Ka. (3) between 
frame k and k+1. 
image frame k+1 
[p:, l2] 
  
  
Fig. 4: Geometry for 3D motion analysis. 
431 
Pin = [Ra] : (Pi - G) * [Rc]- G* T (3) 
T ; 
; x translation 
with Tz T, = ; 
T vector 
G i=N 
x P component 
G = G, = Il = p ; and 
G, * N center 
R, 
[Re] = rotation matrix of rotation vector R = | R, 
R 
>» 
Object rotation can be expressed by a rotation vector 
R = (Rx, Ry, R,)T that describes the successive rota- 
tion of the object around the three axes (x, y, z)" 
parallelto the scene coordinate system centeredat G. 
From this vector the rotation matrix [Rg] is derived 
when the identical matrix [I] is rotated around the 
coordinate axes with Ry first, Ry second and R; last. 
Because [Rg] is derived from the rotation vector R, 
the six parameters of T and R suffice to describe the 
3D object motion. 
The only information available to the analysis system 
is the surface texture projected onto the camera tar- 
get throughout the image sequence. From this se- 
quence the shape and motion parameters have to be 
derived. Assume a scene with an arbitrarily shaped, 
moving textured object observed by a camera C dur- 
ing frames k and k+1 as shown in Fig. 4. The object 
moves between frames k (dashed object silhouette) 
and k+1 (solid object silhouette) according to the gen- 
eral motion equation Eq. (3) with motion parameters 
R and T. A point onthe object surface, called observa- 
tion point Px), holds the surface intensity |, which is 
projected n p, in the image plane at frame k. At 
frame k+1 Py) has moved to P(k+1), still holding 14. In 
image frame k+1 the surface intensity I will now be 
projected at p», whereas the image intensity at point 
p, has changed to Io. 
bi tion R, T 
s motion AR: 
| 
  
object at 
object at 
frame k+1 
frame k