CMRT09: Object Extraction for 3D City Models, Road Databases and Traffic Monitoring - Concepts, Algorithms, and Evaluation 
2 
Figure 1. Road laboratory. Video 
cameras are mounted on the front 
side and on the back side of the car. 
(a) (b) 
Figure 2. Video frame from one camera and a 
corresponding section of rectified road image. 
Figure 3. Lane marking and road defects 
are mapped with a minor user 
interaction. 
The outline of mapping process for a user if the following: 
first automatic detection is applied to a small section of 
rectified road image, after that user checks the results of 
automatic method, corrects the errors if needed and then 
detection algorithm is adapted in order to take new data into 
account. After that user goes on to the following road section 
and the whole procedure is repeated again. Through 
continuous training of detection algorithm with the help of 
operator input error rate of automatic detection decreases; 
thus minimal input is required for accurate mapping. In order 
to reduce user effort during error correction we take 
advantage of hierarchical image segmentation, which helps to 
remove false detections or mark missing objects with just a 
few clicks. 
The paper is organized as follows. Section 2 addresses the 
procedure of data acquisition and transformation of video 
sequences into rectified road images. Section 3 describes 
offline stage of our method. Section 4 gives details on user 
interaction with the system. Our method for lane marking and 
pavement surface defects detection is described in section 5. 
Section 6 is devoted to our machine learning algorithm, 
which helps to tune detectors on various road images 
individually. Experiments on real-world data collected by our 
mobile laboratory are described in section 7. Section 8 is left 
for conclusion and future work. 
2. DATA ACQUISITION 
In this work we have used a vehicle equipped with 4 video 
cameras with resolution 720x576px and Global Positioning 
System (GPS) on board. The cameras capture video of road 
surface and roadside, which can be accurately geographically 
registered by means of GPS. Figure 2 (a) shows an example 
of one frame of video obtained by a video camera mounted 
on a van and corresponding section of rectified road image. 
Although all cameras in capture video, usage of video as 
input for mapping lane marking and road pavement defects 
has severe drawbacks. First, areal objects on road pavement 
surface suffer from projective distortion which degrades 
performance of detection algorithms. For example, 
rectangular pavement patches become trapezoids in video 
frame. Second, some elongated objects are not fully visible 
in any single frame of video sequence. Third, different 
objects are represented with different spatial resolution on the 
same video frame depending on their distance to the camera. 
To overcome these problems we transform video sequence 
into rectified image of the road pavement surface. 
These images are obtained from video using perspective 
plane transformation. Resulting image is one long image in 
the full driven length. All rectified images are stored in raw 
format with time and distance information of all pixels. 
Figure 2 (b) shows an example of video frame obtained from 
one camera and a corresponding section of rectified image of 
road pavement surface. 
As long as image processing algorithms (like image 
segmentation) used at subsequent stages of our workflow are 
memory and time consuming, long rectified road image are 
cut into non-overlapping small sections. Each part is about 
0.5 megapixel image and represents an approximately 5-10 
meters long section of road pavement surface. All these 
section images are further processed in chain, following 
vehicle path. 
3. OFFLINE STAGE OF MAPPING PROCESS 
In our work the mapping workflow consists of two stages: 
off-line and online stage. As long as we aim at interactive 
working time at the time of road mapping, all time- 
consuming operations required by both detection and 
learning are performed off-line. Offline stage happens once 
for each road data before user starts mapping road surface. 
This stage doesn’t require any user assistance. Our detection 
algorithm is based on over-segmentation and classification of 
super-pixels, therefore offline stage includes image 
processing, image segmentation, and calculation of features 
for each image segment. Below these operations are 
described in more details. 
Image processing 
Roadway images are strongly differed to each other in color, 
brightness and texture. This fact substantially complicates the 
detection task. Therefore main goal of image preprocessing is 
to normalize images and put them into some standard state. 
Image processing includes luminance correction, contrast 
adjustment, colour correction and image smoothing. All these 
operations are performed in CIE-Lab colour space. 
For luminance correction we use a modification of Retinex 
algorithm!Land, 1971) . Single-Scale Retinex has artifacts 
such as halos around dark objects and shadows around light 
ones, what damages detection in low-contrast images. 
Conventional Multi-Scale Retinex also has these artifacts 
when it has to deal with strong luminance changes. Since 
most of necessary lightness correction is caused by ruts on 
the road, brightness map is calculated using elongated 
median filter. It helps to reduce halos effect during luminance 
corrections (Figure 4 (b)).