tv
Ine
an
ras
ust
df.
Iti-
nal
Jn.
cle
op
1g.
le,
For
va,
)n.
/R
Fuse, Takashi
A Comparative Study on Gradient-Based Approaches for Optical Flow Estimation
Takashi FUSE, Eihan SHIMIZU" and Morito TSUTSUMI"
" Department of Civil Engineering, University of Tokyo
** Professor, Department of Civil Engineering, University of Tokyo
7" Assistant Professor, Department of Civil Engineering, University of Tokyo
Hongo 7-3-1, Bunkyo-ku, Tokyo, 113-8656
E-mail: <fuse, shimizu, tsutsumi>@planner.t.u-tokyo.ac.jp
JAPAN
KEY WORDS: Optical Flow Analysis, Movement Detection, Object Tracking, Image Sequence, Measurement,
[Image Processing, Computer Vision.
ABSTRACT
The most readily available motion parameter from sequential image is optical flow. Among various optical flow
estimation techniques, gradient-based approach is a common technique. This approach is based on the assumption that
the brightness of a point in the image remains constant during a short time interval, while the location of that point in
the image may change due to motion. This assumption leads to a single local constraint on the optical flow at a certain
point in the image. It is, however, ill-posed as the constraint constitutes only one equation of two unknowns, that is, x-
component and y-component of the flow vector. In order to solve this problem, various methods have been proposed.
There are, however, only a few comparative studies from the viewpoint of the application to the specific and practical
motion analysis. This paper reviews the gradient-based approaches theoretically and compares their performance
empirically from the point of view of application to vehicle motion analysis. The basic methods of gradient-based
approach are reviewed as follows: (1) Increase in the number of observation equations: (a) spatial local optimization
method, (b) temporal local optimization method, (c) multispectral constraints method, (d) second order derivative
method and by their combination; (2) Imposition of a condition: (a) spatial global optimization method, (b) temporal
global optimization method and their combination. The result of empirical comparison shows the difficulty of
estimation of precise and dense optical flow by ordinary gradient-based approaches, when sequential images are taken
at an interval about 1/30 seconds. Hence, it is difficult to analyze vehicle motion by the gradient-based approaches in
this case.
1. INTRODUCTION
Sequential image processing techniques have made progress for motion analysis under the improved performances of
optical sensor and personal computers. Using the sequential image processing techniques, 3D reconstruction and
structure from motion have been attempted, and such attempts will lead to computer vision or robot vision. For 3D
reconstruction and structure from motion, stereo sequential images are employed. Stereo matching requires
displacement vectors at feature points. In order to acquire the displacement vectors, the extraction techniques of the
displacement vectors have been investigated. Optical flow is usually used for this purpose and gradient-based
approach is a common technique for optical flow estimation. Optical flow is the distribution of apparent velocities of
movements of brightness patterns in an image. And gradient-based approach is based on the assumption that the
brightness of a point in the image remains constant during a short time interval, while the location of that point in the
image may change due to motion. This assumption leads to a single local constraint on the optical flow at a certain
point in the image (Horn and Schunck, 1981). Standard computer vision applications require precise and dense optical
flow. In various techniques used for optical flow estimation, gradient-based approach is the most suitable to meet
these requirements. It is, however, ill-posed as the constraint constitutes only one equation of two unknowns, that is,
x-component and y-component of the flow vector. Further constraints are, therefore, necessary to solve for two
unknowns. In order to solve this problem, various constraints have so far been proposed. It is said that these
approaches have large influence upon the result.
On the other hand, there have been only a few comparative studies from the viewpoint of the application to the specific
and practical motion analysis. This paper reviews the gradient-based approaches theoretically and compares their
performance empirically from the point of view of application to vehicle motion analysis.
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B5. Amsterdam 2000. 269