The image displacement vector d = pa — p4 is called 
optical flow vector and describes the projection of the 
observation point displacement P..1)— P(x) onto the 
image plane. When assuming a linear dependency of 
the surface texture between |, and l? and a brightness 
constancy constraint between frame k and k+1 it is 
possible to predict l2 from I4 and its corresponding 
image intensity gradients and hence to estimate d 
from the measurable difference l» — l4. 
T 
vn (3. 5 d 
y 
Ox 
lo is measured at position of p, at frame k+1, where 
l4 is taken from image position p, at frame k. When 
approximating the spatial derivatives as finite differ- 
ences the optical flow vector d = (dy, dy)" can be 
predicted from the image gradients g = (9x, gy)’ and 
the temporal image intensity difference Alpi z]lo-H 
between frame k«1 and k at p, in Eq. (5): 
Ali = g-d = gd. +g d, 
= 84 * (Pax — Dix) + By * Pay — P1y) 
(4) 
(5) 
In Eq. (5) d is related to intensity differences. Substi- 
tuting the perspective projection of P/x) and P(x41) for 
p, and po in Eq. (5) yields a direct geometric to photo- 
metric transform that relates the spatial movement of 
P between frame k and k+1 to temporal intensity 
changes in the image sequence at p1. 
Al = f- g ; Pax = Pa 
" s Poux Pay 
(6) 
+ f. gy Peay = Pay 
y Puri Puy. 
As long as the motion between Py) and P(k+1) can be 
expressed parametrically by a linear equation and 
assuming that the initial position of Py is known, 
P(k,1)can be substituted and the Eq. (6) canbe solved 
through evaluation of the spatial and temporal intensi- 
ty differences at p4. Essentially every surface point 
can be taken as an observation point. It is, however, 
useful to restrict the number of observation points to 
those carrying relevant information. Eq. (6) uses 
image intensity as well as the spatial image gradients. 
Image areas with zero gradient can not be used for 
parameter estimation. A lower bound to the image 
gradient is additionally introduced to account for cam- 
era measurement noise. It is therefore necessary to 
impose a minimum gradient threshold when selecting 
observation points as described by [Hótter, 1988]. For 
each observation point its initial 3D position on the 
object surface, image intensity and spatial image gra- 
dients are recorded. During the analysis each obser- 
vation point is projected and the intensity difference 
to the real image is evaluated. 
Direct estimation of 3D object motion With the 
proposed approach, rigid 3D object motion can be 
432 
estimated directly from the image sequence when the 
object shape is known. Assume an observation point 
with a known position Py that moves in space and is 
observed by a camera C in Fig. 4. The motion is 
governed by the general motion equation (3). Assum- 
ing that rotation between successive images is small, 
the rotation matrix [Rc] can be linearized to Eq. (7). 
à RAR, 
R] = |R 1 -R, 
-R,R, 1 (7) 
rotation matrix [Rg] is linearized to [R'] 
when setting sin® = o and cos - 1 
When substituting [R’] into Eq. (3) a linearized version 
for the general motion equation is found. P(k+1) is 
expressed in explicit form in Eq. (8): 
(k+1)x 
Pası) = |[Pa+yy] = 
Paix 
(k)x — (Pay — GR; + (Paz — G2)R, * T, 
(Pax — GOR, + Py — (Par — GIR, + T, 
— (Page — GIRy + (Po, - Gj)R, + Poy, TT, 
The parameters to be estimated are the translation T 
and the rotation R. When substituting P(x..1) from Eq. 
(8) in Eq. (6) and linearizing the resulting non linear 
equation through Taylor expansion for R and T at R - 
T « 0, the linearized equation for a single observation 
point Py is computed as 
Alpi f: gx/ Pz : Ty 
f:gy/ Pz Ty 
f-( Pxgx + Pygy) / Pz? Tz 
f: [ Pxgx(Py — Gy) + Pygy(Py — Gy) 
Pzgy(Pz — Gz) ]/ P,2 - Ry 
f-[ Pygy(Px— Gx) + Pxgx(Px - G3) 
P;g«(Pz - Gz) ]/ PZ? - Ry 
f: [gx(Py - Gy) - gy(Px- Gg ]/Pz Rz 
(8) 
+ 
+ 4 "| 
with Pay = (Px Py, PT. (9) 
i imation Atleast six 
distinctive observation points that lead to six linear 
independent equations are needed to solve forthe six 
motion parameters R and T. In real imaging situations 
the measurements of the spatial and temporal deriva- 
tives are noisy and some of the observation points 
selected may be linear dependent of each other. To 
cope with those conditions an over constrained set of 
equations is established and a linear regression is 
carried out using least squares fit. All observation 
points of one object are evaluated. It is important to 
note that we do not measure optical flow locally and 
then try to combine the flow field. Instead all observa- 
tion points of a rigid surface are used to solve for R 
and T. To account for the linearizing, the estimation is 
iterated. The position P of each observation point is 
initially determined by object shape and position. An