REALISTIC 3D VIEW GENERATION USING DEFORMAL STEREO MATCHING 
Jae-Chul Kim, Jong-Hyun Park 
LBS Research Team, Telematics Research Division, ETRI 
161 Kajong-Dong, Yusong-Gu, Taejon, 305-350, Republic of Korea 
kimje@etri.re.kr 
KEY WORDS: Vision Sciences, Matching, Pattern, Recognition, Detection 
ABSTRACT: 
In this paper, we propose an adaptive stereo matching algorithm to encompassing stereo matching problems in projective distortion 
region. Since the projective distortion region can not be estimated in terms of fixed-size block matching algorithm, we tried to use 
adaptive window warping method in hierarchical matching process to compensate the perspective distortions. In addition, 
probability theory was adopted to encompass uncertainty of disparity of points over the window in this study. The proposed stereo 
matching algorithm has tested on both disparity map and 3D model view. The experimental result shows that remarkable 
improvement is obtained in the projective distortion region. 
1. INTRODUCTION 
Stereo images are obtained from the different perspective 
position(Figurel), so that each image can have the effect of 
projective distortion as shown in Figure 2. 
object 
   
direction of 
left view 
  
  
   
direction of 
right view 
  
  
epipolar plane 
   
      
   
left image 
I 
baseline 
    
ve Sd epipolar line 
dr = 
E 
right image 
Figure 1. Stereo geometry with parallel axes. 
Since the surface of a real object is projected on the right and 
left cameras, each projected-image has different viewing 
characteristics (Note Figure2 show that area of dr is larger than 
that of dl). A main problem of stereo matching heavily depends 
upon selecting an appropriate window size. However, same 
window size, both reference window and searching window, 
does not appropriate for projective distortion region. It is 
necessary to consider projective distortion in stereo matching 
process. Due to above result, many researcher (T. Kanade, 
1994; Z. F. Wang, 1994; P. N. Belhumeur, 1992) have been 
interested in the effect of projective distortion in recent years. 
Kanade (T. Kanade, 1994) presented an adaptive window 
method to reduce the effect of projective distortion. His method 
employs a statistical model of the disparity distribution within a 
window. By evaluating the local variation of the intensity and 
the disparity, the method can select an appropriate window size 
768 
and estimate disparity with the least uncertainty for each pixel 
of an image. Wang and Ohnishi (Z. F. Wang, 1994) presented a 
3D-to-3D method. Instead of using a statistical model of the 
disparity distribution, the method uses a geometric model of the 
matching process (G. Medioni, 1985). 
     
    
focus line : 
left meee Yee right 
camera camera 
Figure 2. Projective distortion. 
A transformational template-based matching method is used to 
recover the depth data from the projective distortion 
information. There has been no study that tried to use window 
warping technique (G. Wolberg, 1994) in the projective 
distortion. 
The algorithm we propose is based on wondow warping 
technique. Furthermore Fant’s algorithm (G. Wolberg, 1994) is 
used in order to warp a local mask in hierarchical block- 
matching process. 
2. WINDOW WARPING ALGORITHM 
2.1 Projective Distortion 
Since surface normal is often very tilted with respect to the 
optical axes of camera, the projected stereo images has 
projective distortion. The evidence of the projective distortion 
can be found in the comparison of line profiles (Figure 3). In 
Figure 3, Iz and /, (270th line profiles) are a line profile of 
right and left image. These are give a good account of 
projective distortion phenomenon in ‘man’ stereo images. 
Int 
Fis 
col 
lef 
of 
Th 
in 
ste 
Ka 
ma 
an 
col 
wn 
prc 
2.2