ns 
ts of the left 
h map for each 
anent road signs 
the normalized 
e und saturation 
varying lighting 
nobile mapping, 
the HSV color 
to restrict the 
by applying a 
the detection of 
tio from 0.06 to 
1age acquisition 
d and classified 
ad yellow color 
r the hue and 
ned empirically 
paigns. For blue 
52 and 0.72 and 
; featuring a hue 
ion value which 
covering yellow 
corresponds to 
ed by the two 
(1) 
(2) 
  
learea 
rdized road sign 
the dense depth 
ts and widths of 
also utilized in 
ons with similar 
planar segment 
and 4j) and the 
ves as detection 
rocess. 
  
Figure 4. Automated detection of a road sign with predominantly yellow colors (a: left normalized image, b: hue component, 
c: saturation component, d: distance reduced hue component, e: distance reduced saturation component, f: yellow color segments, 
g: depth map, h: distance reduced depth map, i: planar segments, j: planar segments after morphological operations) 
3.2 Automated classification and mapping of road signs 
The classification process for a detected road sign is performed 
using  cross-correlation-based template matching with 
predefined reference templates. Since the hierarchical 
classification approach uses the properties color and shape to 
considerably reduce the candidate set, not all road signs have to 
be tested. Road signs which are inexistent on the captured roads 
can also be excluded from the classification process. 
The template with the highest normalized cross-correlation 
coefficient within the search image is determined. This value 
also serves as classification indicator. If it exceeds a predefined 
threshold, the classification is considered as successful. 
Candidates with a detection or classification indicator below 
this predefined threshold are assigned to a list of uncertain 
objects for a subsequent user-controlled verification and (re-) 
classification. 
The dimensions of the search image are defined by the road sign 
dimensions in the normalized image plus a margin on each side 
(e.g. 10 pixels). The template is scaled to the dimensions of the 
color segment within the search image. The correlation is 
computed based on the channel which empirically showed the 
highest similarity between a real road sign image and the 
corresponding synthetic template. This is the blue channel for 
red road signs, the red channel for blue signs and the saturation 
component for yellow road signs. Pixels of the search image 
which are white in the template have a too low gray value 
depending on the image quality. Hence, to improve the 
matching results, all pixel values are set to white (maximal 
value) or black (zero). The required threshold is computed 
dynamically based on the gray value distribution of the search 
image. Further details can be found in Cavegn & Nebiker 
(2012). 
When a road sign could be detected and classified, the 3D 
object coordinates of the sign are determined. For the 
computation of the model coordinates, the image coordinates of 
the sign's center of gravity, the corresponding depth value as 
well as the parameters of the interior orientation are needed. 
The following transformation to the desired geodetic reference 
61 
system requires that the exterior orientation parameters of the 
left normalized image are known. 
The detection, classification and mapping processes 
automatically yield a number of attribute data like the 3D 
coordinates, the template number and the standardized side 
lengths. They can further be used for creating or updating a GIS 
database. This is essential, because even in highly developed 
countries, GIS-based digital road sign inventories either do not 
yet exist at all or were derived from analogue maps and are 
normally not up-to-date. 
4. INVESTIGATIONS AND RESULTS 
The implemented algorithms were evaluated based on two field 
test campaigns in the city of Muttenz near Basel with the 
stereovision-based mobile mapping system by the FHNW 
Institute of Geomatics Engineering (IVGI). Currently, this 
MMS features two pairs of stereo systems, each with a stereo 
base of approximately 90 cm, and with industry cameras at 
different geometric resolutions (Full HD and 11MP). Direct 
georeferencing of the stereo imagery is provided by an entry- 
level GNSS/IMU system in combination with a distance 
measuring indicator. Earlier empirical tests of the IVGI MMS in 
multiple test campaigns demonstrate accuracies in object 
coordinate space for well-defined points of 3-4 cm in along- 
track and cross-track and 2-3 cm in vertical dimension — under 
presence of a good GNSS availability (Burkhard et al. 2012). 
The first test campaign was carried out in winter time 
(November 2010) with difficult to poor lighting conditions, the 
second in summer (July 2011) in sunny conditions. In both 
cases, about 2500 stereo image pairs were captured on 
residential roads at five frames per second and at a driving 
speed of approximately 40 km/h resulting in about one Full HD 
stereo frame every two meters. For the subsequent evaluation of 
the detection and classification quality, all relevant road signs 
with predominantly red, blue and yellow colors were identified. 
These relevant signs were all road signs adjacent to the driving 
lane on the right-hand side facing the driver, i.e. with a road 
sign plane roughly perpendicular to the road axis, thus covering