conclude that differential techniques, such as the local 
weighted least squares method proposed by Lucas and 
Kanade (1981) perform best in terms of efficiency and 
accuracy. Phase-based methods (Fleet and Jepson, 
1990) show slightly better accuracy but are less effi- 
cient in implementation and lack a single useful con- 
fidence measure. Bainbridge-Smith and Lane (1997) 
come to the same conclusion comparing the perfor- 
mance of differential methods. Performing analytical 
studies of various motion estimation techniques, Jähne 
(1993, 1997) showed that the three-dimensional struc- 
ture tensor technique yields best results with respect 
to systematic errors and noise sensitivity. This could 
be verified by Jähne et al. (1998), analyzing a cal- 
ibrated image sequence with ground truth data pro- 
vided by Otte and Nagel (1994). 
3. LOCAL WEIGHTED LEAST SQUARES 
Lucas and Kanade (1981) propose a local weighted 
least squares estimate of the constraint (1) on indi- 
vidual pixels within a local spatial neighborhood U 
by minimizing: 
/ Ax = x) (Vag) E +g) dx (2) 
with a weighting function h(x). In practical imple- 
mentations the weighting is realized by a Gaussian 
smoothing kernel. Minimizing (2) with respect to the 
two components f; and f5 of the optical flow f yields 
the standard least squares solution 
ow eniin] "len] " 
A f b 
with the abbreviation 
(a) = / h(x — x')adx'. (4) 
The solution of (3) is given by f = A7!b, provided 
the inverse of A exists. If all gradient vectors within 
U are pointing into the same direction, A gets singu- 
lar and the aperture problem remains within the local 
neighborhood. These cases can be identified by ana- 
lyzing the eigenvalues of the symmetric matrix A prior 
to inversion (Barron et al., 1994, Simoncelli, 1993) 
and only the normal flow f, is computed from (1) by 
fi = —g/||Vgl. It is, however, a critical issue to 
obtain information about the presence of an aperture 
problem from the numerical instability of the solution. 
Thresholds on the eigenvalues proposed by Barron et 
al. (1994) and Simoncelli (1993) have to be adapted 
to the image content which prevents a versatile imple- 
mentation. 
706 
  
  
  
  
  
Figure 2: Illustration of the spatiotemporal brightness 
distribution of moving patterns. The sequence shows in- 
frared images of the ocean surface moving mainly in posi- 
tive x-direction. 
Jähne (1997) shows that an extension of the integra- 
tion in (2) into the temporal domain yields a bet- 
ter local regularization if the optical flow is modeled 
constant within the spatiotemporal neighborhood U. 
This does, however, not change the minimization pro- 
cedure and results in the same linear equation system 
(3) with spatiotemporal integration of the individual 
components (a). 
From a probabilistic point of view, the minimization of 
(2) corresponds to a maximum likelihood estimation 
of the optical flow, given Gaussian distributed errors 
at individual pixels. Black and Anandan (1996) show 
that the Gaussian assumption does not hold for mo- 
tion discontinuities and transparent motions. By re- 
placing the least squares estimation with robust statis- 
tics they come up with an iterative estimation of mul- 
tiple motions. 
4. STRUCTURE TENSOR APPROACH 
The displacement of gray value structures within con- 
secutive images of a sequence yields inclined structures 
with respect to the temporal axis of spatiotemporal 
images (Figure 2). 
The orientation of iso-grey-value lines within a 
tree-dimensional spatiotemporal neighborhood U can 
mathematically be formulated as the direction r = 
[ri r2, ra]? being as much perpendicular to all grey 
value gradients Vg in U as possible. The direction r 
and the optical flow f are related by f — r7! [ri ra]T 
With the definition of r, (1) can be formulated as: 
h 
[9c 9) 9] E 97] era (Pg) r0, 20 (8) 
1 
where V4g denotes the spatiotemporal gradient vec- 
tor Vig — [gs gy, 91]. - It is important to note that 
(1) and (5) are mathematically equivalent formula- 
tions and no constraint is added by extending the 
formulation of the brightness change constraint into 
three-dimensional space. 
The : 
whic. 
quad 
with 
The 
Agai 
time 
the 
(Jàh 
In o 
strai 
by t 
solu: 
vect 
the 
orth 
mat 
mat 
It is 
leas! 
the : 
the 
sion 
imiz 
only 
two 
vari 
tor 
4.1 
The 
carr 
ing 
a th 
pro 
stru 
wit] 
and