In: Paparoditis N., Pierrot-Deseilligny M.. Mallet C... Tournaire O. (Eds). 1APRS. Vol. XXXVIII. Part 3A - Saint-Mandé, France. Septeniber 1-3. 2010 
MATCHING TERRESTRIAL IMAGES CAPTURED BY A NOMAD SYSTEM TO IMAGES 
OF A REFERENCE DATABASE FOR POSE ESTIMATION PURPOSE 
Arnaud Le Bris and Nicolas Paparoditis 
Université Paris-Est, IGN. Laboratoire MATIS 
73 Avenue de Paris 
94165 SAINT-MANDE Cedex - FRANCE 
aniaud.le-bris@ign.fr, nicolas.paparoditis@ign.fr 
http://recherche.ign.fr/labos/matis/ 
Commission III, WG III/3 
KEY WORDS: Registration. SIFT, tie points, facades, urban terrestrial image database 
ABSTRACT: 
Mobile mapping systems have been developed to achieve a fast automated acquisition of huge quantity of georeferenced terrestrial 
images in urban cities. Stereopolis is such a system making it possible to capture panoramic groups of images. These georeferenced 
photos are then stored in urban images street scale reference databases. 
The issue investigated in this paper is the problem of tie points extraction between new images captured with an approximate georef- 
erencement by a "nomad system" and images from the reference database in order to estimate a precise pose for these new images. 
Because of several difficulties (diachronism. viewpoint change, scale variation, repetitive patterns) extracting enough correct well dis 
tributed tie points is difficult and directly extracted and matched SIFT keypoints from original images are most of the time not sufficient. 
Nevertheless, results can be improved using ortho-rectified images on the facade plane instead of original images. Rectification param 
eters ( 3D rotation) are obtained from the coordinates of vanishing points corresponding to the two main directions of the facade. These 
points can indeed be extracted from linear features of the facade on the images. However, many point matches remain false and difficult 
to detect using only their image coordinates. The use of both image coordinates and scale and orientation associated to the matched 
SIFT keypoints makes it possible to detect outliers and to obtain an approximate similitude model between the two ortho-images. A 
more accurate model can then be computed from correct tie points. 
1 INTRODUCTION 
1.1 Context 
These last years, mobile mapping systems have been designed 
and developed to achieve a fast automated acquisition of huge 
quantity of georeferenced terrestrial images in urban cities. Stere 
opolis is such a system. It is a vehicle surmounted by a crown of 
cameras, making it possible to capture panoramic groups of im- 
ages.These georeferenced photos are then stored in urban images 
street scale reference databases and can be used for several appli 
cations or simply displayed ow ing to special applications such as 
for instance (iTowms. last visited on the 1st of March 2010) mak 
ing it possible to navigate within the image flow (in panoramic 
geometry) and exploit features extracted from images to provide 
high semantic level data. 
Tools could now' be developed to offer the possibility for a user 
to georeference its own images from the database images. Such 
tools could then be used for several applications, such as : 
• enrich the database : images with better resolution than the 
ones from the database are captured to enrich the database, 
especially for monuments. 
• update the database : new images are captured punctually to 
update the database, (for example, photos of the facade of a 
new shop) • 
• image based location : an image is taken by a user with a 
mobile phone + camera + GPS (+ INS), and sent to the tool. 
It is then matched to the database and georeferenced. 
These new images have not necessarily been captured by a pre 
cise georeferencing image acquisition system. Therefore, they 
have to be precisely located to be added to the database (or to 
an additional database) and their pose have to be coherent with 
the one of the images of the database. Nevertheless, it can be 
assumed that an approximate pose is available. Such information 
can indeed have been obtained either directly from the acquisition 
system - for example a mobile phone + camera + GPS (+INS) - 
or by the user (approximate location indicated on a map, name of 
the street...). As a consequence, the main issue in this paper is not 
to find matching images in the whole database for the new pho 
tos but to extract tie points between them and the images of the 
database in order to be able to georeference them. Nevertheless, 
tie points between these tw'o sets of images are not easy to detect 
and several difficulties are encountered (as explained in section 
2). 
1.2 Input data 
It must here be said that no real data captured by a ’’nomad sys 
tem’’ - such as a mobile + camera + GPS (+INS) - was avail 
able and that the methods have been tested on images captured 
by a standard digital camera (without georeferencing system). 
The only information about georeferencement is the name and 
the side of the street. 
On the other hand, images of the database are precisely georefer 
enced. They have indeed been captured by the mobile mapping 
system Stereopolis. 
1.3 SIFT keypoint matching 
In this paper, SIFT is used to extract tie points between the new 
photos and images from the database. SIFT (Scale Invariant