°ntal 
  
Juliang Shao 
  
MULTI-IMAGE MATCHING USING SEGMENT FEATURES 
Juliang SHAO', Roger MOHR' and Clive FRASER* 
*Department of Geomatics, The University of Melbourne 
Victoria 3052, Australia 
Email: jsha@sli.unimelb.edu.au & c.fraser@eng.unimelb.edu.au 
#Xerox Research Centre Europe, 6 chemin de Maupertuis 
F- 38240 Meylan, France 
Email: Roger.Mohr@xrce.xerox.com 
Working Group III/1 
KEY WORDS: feature-based matching, computer vision, multi-image matching, quality control, segment features. 
ABSTRACT 
This paper presents a strategy for matching features in multiple images, which emphasises reliable matching and the 
recovery of feature extraction errors. The process starts from initial ‘good’ matches, which are validated in multiple 
images using multi-image constraints. These initial matches are then filtered through a relaxation procedure and are 
subsequently used to locally predict additional features that might well be extracted using different thresholds for the 
feature extraction process. The relaxation labelling is simultaneously performed among multiple images, as opposed to 
the usual case of just two images. The overall process has been applied to segment matching of both aerial and close- 
range imagery and an example application is briefly reported. 
1 INTRODUCTION 
In parallel with advances in digital imaging, the photogrammetric and computer vision communities have witnessed 
significant developments in image matching over the past two decades (eg Gruen, 1985; Dhond & Aggarwal, 1989; 
Faugeras, 1993 and Jones, 1997). Nevertheless, image matching remains a bottleneck in both the automation of 3D 
information extraction from imagery and in the browsing of image databases (Jain and Vailaya, 1996). Within 
photogrammetry, image matching, and typically feature-based matching, is principally applied in the recovery of 
surface contour information from stereo pairs of aerial photographs, where the matching can be either on selected, 
sparsely distributed feature points or on a dense array comprising all available feature points. Feature-based matching 
can also be readily extended to multiple images (>2) and in this paper we consider the use of line segments as features 
to support multi-image matching. 
The process of matching features in multiple images exhibits a number of fundamental characteristics. Of these, we 
focus in this paper upon reliable matching and the recovering of feature extraction errors. The operation starts from 
initial ‘good’ matches, which are validated within the multiple images using applicable constraints. These initial 
matches are then filtered through a relaxation procedure and used to locally predict further features which might be 
extracted using different thresholds for the feature extraction process. A relaxation labeling, which is an extension of the 
more typical two-image case, is then simultaneously performed among the multiple images. The different steps lead, 
for a local distribution of features, to an integration of signal (intensity) similarity and geometric constraints. 
The proposed multi-step approach has been adopted in order to overcome both ambiguities in matching and errors in 
feature extraction. As regards the issue of errors, the approach will be illustrated through the use of features comprised 
of contour segments. In this case, errors in segment extraction are partially a function of both spurious segments and 
broken line segments. 
The first step in the process is to collect an initial set of reliable features, using a strict threshold for the feature 
extraction algorithm, namely a strong gradient for the segments. These segment features are then matched using two 
types of information. The first is geometrical information (Gruen, 1985) within the multiple images, which might 
comprise epipolar lines for stereo images for instance. For the case of three images, however, the position of interest 
points is further constrained since a match in two images will have a single correspondence in the third. This 
  
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000. 837