AUTOMATIC EDGE MATCHING 
ACROSS AN IMAGE SEQUENCE BASED ON RELIABLE POINTS 
Yixiang Tian a ’ b ' *, Markus Gerke a , George Vosselman a , Qing Zhu b 
1 International Institute for Geo-Information Science and Earth Observation (ITC), Hengelosestraat 99, P.O.Box 6, 
7500AA, Enschede, the Netherlands - (ytian, gerke, vosselman)@itc.nl 
b State Key Lab of Information Enginneeming in Surveying Mapping and Remote Sensing, Wuhan University, 129 Luo 
Yu Road, Wuhan, Hubei, 430079, P.R. China - zhuq66@263.net 
ICWG III/V 
KEY WORDS: Image sequence, Video, Edge, Matching, Analysis, Reconstruction 
ABSTRACT: 
This paper presents a new method for matching edges across a video image sequence. The method can deal with uncalibrated images 
acquired with a hand held camera. Compared to previous work, the method employs geometric constraints between edges based on 
reliable matched points, which reduces the search space for corresponding 2D edges in the frames. The 3D edge parameters are 
estimated from these matched 2D edges by using the Gauss-Markoff model with constraints. End points of each 3D edge are found 
by analyzing the end points of the corresponding 2D edges. The results show that the developed algorithms are able to efficiently 
and accurately reconstruct 3D edges from image sequences. 
1. INTRODUCTION 
Modelling 3D objects and scenes from image sequences is a 
research topic since several years [Baltsavias, 2004; Pollefeys, 
2004; Remondino and EL-Hakim, 2006]. The high overlapping 
of images within a video sequence lead to highly redundant 
information, which is exploited for tie point extraction, feature 
tracking and 3D object extraction. However, the short baseline 
between the images also leads to a poor ray intersection 
geometry and thus to mismatches across the sequence. 
At the same time, using edge information for the reconstruction 
of man-made objects from images has been concerned by 
researchers from the fields of photogrammetry and computer 
vision for a long time [Hartley and Zisserman, 2000]. As the 
point clouds obtained from applying feature and camera 
tracking steps to a video sequence are not dense enough, which 
do not allow a complete description of 3D scene and not all 
import points for object reconstruction, such as comer points, 
can be extracted. Edge features can provide more constraints 
about objects’ shape than point features. Existing approaches 
for edge detection can obtain acceptable result [Canny, 1986; 
Meer and Georgescu, 2001]. Nevertheless, edge matching is 
still a difficult problem for several reasons. One is that edges 
belonging to the same entity in object space are often extracted 
incompletely and inaccurately in the single images of the 
sequence. Sometimes, an ideal edge might be broken into two 
or more small segments that are not connected to each other. 
Further, the end points are not reliable, and even with a correct 
orientation, it is difficult to build up topological connections 
between edges. A second reason for the complexity of edge 
matching is due to the fact that there is no strong 
disambiguating geometric constraint available over two or more 
views during edge matching. There is only a weak overlap 
constraint for edge segments of finite length arising from 
applying epipolar geometry constraint to end points [Schmid 
and Zisserman, 1997; Baillard et al., 1999] . 
Existing approaches to edge matching in the literature are 
generally categorized into two types. One is matching 
individual edges between images based on a similarity measure. 
The similarity measure is based on the comparison of edge 
attributes, such as orientation, edge support region information. 
The other strategy is structural matching, which considers more 
geometrical and topological information among edge features. 
But this kind of methods often have a high complexity and they 
are sensitive to error in the segmentation process [Armstrong 
and Zisserman, 1995; Baillard et al., 1999; Kunii and Chikatsu, 
2004; Klein and Murray, 2006]. When camera projection 
information is available, matching individual edges can get 
precise and efficient results and reduce the complexity and 
computation time. 
Most edge matching methods are based on stereo or triplet 
image pairs, and the results are merged together if there are 
more images [Baillard et al., 1999; Zhang et al., 2005]. How to 
use redundant information from video image sequence for edge 
matching and how to eliminate errors caused by short base line 
when reconstructing 3D edge geometry are the main task of this 
paper. All the problems mentioned above are considered in this 
method. Each step considered in this method will be explained 
in the following paragraphs. 
In section 2 our preprocessing steps on feature extraction are 
described. The main method is explained in section 3, divided 
to four parts (overview, point quality analysis, 3D edge 
estimation and end points decision). Results are shown in 
section 4 and discussed in section 5, which also indicates some 
further work. 
2. PREPROCESSING 
2.1 Feature Presentation 
* Corresponding author. 
657