10vement 
' moving 
of three- 
obtained 
imate the 
ments in 
A robust 
ough the 
Artificial 
tation. of 
Oc.IEEE, 
ters from 
ia Sinica 
of China 
ibul 2004 
  
  
A NEW APPROACH FOR AUTOMATIC SELECTION AND TRANSFER OF 
CORRESPONDING IMAGE POINTS IN DIGITAL VIDEO IMAGE SEQUENCES 
J.-R. Tsay * I.-C. Lee 
? Dept. of Geomatics, Natl. Cheng Kung University, No. 1, University Rd., 701 Tainan, Taiwan - 
tsayjr@mail.ncku.edu.tw 
? EverTime Technology, Inc., 3F, No. 107, Sec. 2, Roosevelt Rd., 106 Taipei, john@evertime.com.tw 
Commission III, IC WG IIS 
KEY WORDS: Digital, Video, Close Range, Tracking, Registration, Algorithms 
ABSTRACT: 
We present a new approach for automatic selection and transfer of corresponding image points in digital video (DV) image 
sequences. This approach utilizes and extends some well-known techniques and theories (e.g. the optical flow theory) as well as a 
designed simple error-detection mechanism to achieve a much better efficiency than the well-known Lucas-Kanade optical flow 
estimation (LK) method. Our test results use the SONY DCR-PC115 DV image sequences and show that the trackable range of 3-4 
pixels in the LK method can apparently be enlarged to 30 pixels in this new approach. The proposed error-detection mechanism 
simply utilizes the average gradient, normalized cross-correlation, and a simple image registration aided by least squares adjustment. 
Test results show that it can efficiently detect and delete wrong tracked points, and thus improve the quality of point transfer, and 
provide accurate coordinates of a large number of corresponding image points in DV image sequences. This work aims at high 
precision automatic image triangulation for the automatic real-time mobile mapping vehicle system (MMVS). Some future works 
need to be done. 
1. INTRODUCTION 
In the field of geomatics, automatic real-time mobile mapping 
vehicle system (MMVS) is being developed for the real-time 
digital-map-updating and other surveying purposes. In that 
system, those techniques for automatic image point extraction, 
point transfer, and point measurement are useful, and 
occasionally necessary. Therein, corresponding points in 
terrestrial or close-range stereo images can be found in a full- or 
semi-automatic manner by different image matching techniques 
or by differential methods (Trucco & Verri, 1998). 
[Image matching techniques may utilize low-level data such as 
raw image values, middle-level data such as edge features, or 
high-level data such as symbolic and topological relationships. 
Also, they are often integrated with other disciplines and 
applied in the so-called wide-baseline image matching and 
tracking. For example, a least squares matching tracking 
algorithm was also proposed for human body modeling in 
(D'Apuzzo et al, 2000). The normalized cross correlation 
(NCC) method was used for image tracking and positioning 
targets at sea by using DV images taken on a helicopter in 
(Chen & Chen, 2002). The Kalman filter was applied in (Hong, 
2002) to predict matching areas and thus to decrease the size of 
searching window of correlation-based tracking. (Nguyen et al., 
2001) also adopted the Kalman filter to develop a new method 
for occlusion robust adaptive template tracking. (Hartley & 
Zisserman, 2000) incorporates the epipolar geometry into the 
random sample consensus (RANSAC) method proposed in 
(Fischler & Bolles, 1981) for efficient and reliable wide- 
baseline image matching. Nevertheless, automatic wide- 
baseline image matching techniques still need to be further 
studied (Pritchett & Zisserman, 1998; Van Gool et al., 2002). 
Differential methods such as the so-called optical flow approach 
are suitable for tracking corresponding points in a series of DV 
images, where two consecutive images have a very short 
baseline. Gradient-based optical flow estimation approaches are 
simple techniques for tracking image sequences (Lim & El 
Gamal, 2001). In 1981, two famous approaches are proposed: 
the LK method and the Horn-Schunck optical flow (H-S) 
method. The LK method computes most easily and fast, but it 
can track only those short image displacement vectors. It is 
often adopted in digital signal processors as well as in the 
devices of high image frame rate. The H-S method computes 
slowly and iteratively. It is suitable for the image displacement 
vector field with local continuous smooth variation. Under the 
circumstances, the LK method will be adopted and improved in 
our approach. 
In this paper, a newly developed approach, called “iterated 
optical flow estimation (IOFE)” approach, is used to improve 
the function of traditional optical flow method. Also, both 
optical flow method and normalized cross-correlation (NCC) 
method are to be compared concerning their effectiveness and 
quality. 
2. THE NOVEL APPROACH 
In order to simplify the problem to be solved, it is assumed that 
only those sequential DV images of immovable objects in a 
scene are to be taken by a DV camera in a moving mode. Thus, 
our approach is expected to provide photo coordinates of a large 
number of corresponding image points of immovable object 
points to photo triangulation application in a MMVS. Photo 
triangulation determines then 3D object coordinates of all pass 
points and orientation parameters of each image, where all pass 
points are assumed to be immovable and to have fixed 3D 
object coordinates. These data can be further used to reconstruct