tv 
Ine 
an 
ras 
ust 
df. 
Iti- 
nal 
Jn. 
cle 
op 
1g. 
le, 
For 
va, 
)n. 
/R 
Fuse, Takashi 
  
A Comparative Study on Gradient-Based Approaches for Optical Flow Estimation 
Takashi FUSE, Eihan SHIMIZU" and Morito TSUTSUMI" 
" Department of Civil Engineering, University of Tokyo 
** Professor, Department of Civil Engineering, University of Tokyo 
7" Assistant Professor, Department of Civil Engineering, University of Tokyo 
Hongo 7-3-1, Bunkyo-ku, Tokyo, 113-8656 
E-mail: <fuse, shimizu, tsutsumi>@planner.t.u-tokyo.ac.jp 
JAPAN 
KEY WORDS: Optical Flow Analysis, Movement Detection, Object Tracking, Image Sequence, Measurement, 
[Image Processing, Computer Vision. 
ABSTRACT 
The most readily available motion parameter from sequential image is optical flow. Among various optical flow 
estimation techniques, gradient-based approach is a common technique. This approach is based on the assumption that 
the brightness of a point in the image remains constant during a short time interval, while the location of that point in 
the image may change due to motion. This assumption leads to a single local constraint on the optical flow at a certain 
point in the image. It is, however, ill-posed as the constraint constitutes only one equation of two unknowns, that is, x- 
component and y-component of the flow vector. In order to solve this problem, various methods have been proposed. 
There are, however, only a few comparative studies from the viewpoint of the application to the specific and practical 
motion analysis. This paper reviews the gradient-based approaches theoretically and compares their performance 
empirically from the point of view of application to vehicle motion analysis. The basic methods of gradient-based 
approach are reviewed as follows: (1) Increase in the number of observation equations: (a) spatial local optimization 
method, (b) temporal local optimization method, (c) multispectral constraints method, (d) second order derivative 
method and by their combination; (2) Imposition of a condition: (a) spatial global optimization method, (b) temporal 
global optimization method and their combination. The result of empirical comparison shows the difficulty of 
estimation of precise and dense optical flow by ordinary gradient-based approaches, when sequential images are taken 
at an interval about 1/30 seconds. Hence, it is difficult to analyze vehicle motion by the gradient-based approaches in 
this case. 
1. INTRODUCTION 
Sequential image processing techniques have made progress for motion analysis under the improved performances of 
optical sensor and personal computers. Using the sequential image processing techniques, 3D reconstruction and 
structure from motion have been attempted, and such attempts will lead to computer vision or robot vision. For 3D 
reconstruction and structure from motion, stereo sequential images are employed. Stereo matching requires 
displacement vectors at feature points. In order to acquire the displacement vectors, the extraction techniques of the 
displacement vectors have been investigated. Optical flow is usually used for this purpose and gradient-based 
approach is a common technique for optical flow estimation. Optical flow is the distribution of apparent velocities of 
movements of brightness patterns in an image. And gradient-based approach is based on the assumption that the 
brightness of a point in the image remains constant during a short time interval, while the location of that point in the 
image may change due to motion. This assumption leads to a single local constraint on the optical flow at a certain 
point in the image (Horn and Schunck, 1981). Standard computer vision applications require precise and dense optical 
flow. In various techniques used for optical flow estimation, gradient-based approach is the most suitable to meet 
these requirements. It is, however, ill-posed as the constraint constitutes only one equation of two unknowns, that is, 
x-component and y-component of the flow vector. Further constraints are, therefore, necessary to solve for two 
unknowns. In order to solve this problem, various constraints have so far been proposed. It is said that these 
approaches have large influence upon the result. 
On the other hand, there have been only a few comparative studies from the viewpoint of the application to the specific 
and practical motion analysis. This paper reviews the gradient-based approaches theoretically and compares their 
performance empirically from the point of view of application to vehicle motion analysis. 
  
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B5. Amsterdam 2000. 269