In: Stilla U, Rottensteiner F, Paparoditis N (Eds) CMRT09. IAPRS, Vol. XXXVIII, Part 3/W4 — Paris, France, 3-4 September, 2009 
1 
EFFICIENT ROAD MAPPING VIA INTERACTIVE IMAGE SEGMENTATION 
O. Barinova, R. Shapovalov, S. Sudakov, A. Velizhev, A. Konushin 
Moscow State University, Dept, of Computational Mathematics and Cybernetics 
{obarinova, Shapovalov, ssudakov, avelizhev, ktosh}@graphics.cs.msu.ru 
Commission III, WG III/5 
KEY WORDS: Automation, Video, Processing, Incremental, Learning, Object, Detection 
ABSTRACT: 
Last years witnessed the growth of demand for road monitoring systems based on image or video analysis. These systems usually 
consist of a survey vehicle equipped with photo and video cameras, laser scanners and other instruments. Sensors mounted on the 
van collect different types of data while the vehicle goes along the road. Recorded video can be geographically referenced with the 
help of global positioning systems. Road monitoring systems require special software for data processing. This paper addresses the 
problem of video analysis automation, and particularly the pavement monitoring functionality of such mobile laboratories. We show 
that computer vision methods applied to this problem help to reduce amount of manual labour during data analysis. Our method 
transforms video collected by mobile laboratory into rectified geo-referenced images of road pavement surface, and allows mapping 
of lane marking and road pavement defects with minimum user interaction. In our work the mapping workflow consists of two 
stages: off-line and online stage. In order to reduce user effort during error correction we take advantage of hierarchical image 
segmentation, which helps to delete false detections or mark missing objects with just a few clicks. Through continuous training of 
detection algorithm with the help of operator input error rate of automatic detection decreases; thus minimal input is required for 
accurate mapping. Experiments on real-world road data show effectiveness of our approach. 
1. INTRODUCTION 
Roadway monitoring systems are widely-used for supervising 
road pavement surface and repair planning. These systems 
usually include a complex of video cameras and other sensors 
mounted on a car as shown on Figure 1. The sensors record 
road pavement surface when travelling on a pavement at 
traffic speed. 
Most existing software for road monitoring involves manual 
processing of video collected by these mobile laboratories. 
Operator manually marks objects like lane marking and 
pavement surface defects (potholes, cracking and patches) on 
each video frame. This procedure is laborious and takes 
plenty of time; therefore the task of automation of objects 
detection comes into focus. In this paper we consider the 
problem of automation of video analysis for pavement 
surface monitoring. We describe a tool which assists in 
utilising visual observation data of pavement surface and 
mapping lane marking and pavement surface defects. 
Our main goal is to minimize effort of operator at the time of 
mapping lane marking and road defects while preserving 
accuracy of mapping result. The effectiveness of our method 
is achieved by intensive usage of computer vision techniques 
together with user-friendly interface that allows checking 
results of automatic detection and correcting errors if needed. 
As long as direct mapping of lane marking and road 
pavement defects in video sequences faces severe difficulties, 
we transform video into rectified images of road pavement 
surface. These images are further processed during interactive 
mapping. 
While to our knowledge there haven’t been much research on 
topic of road defects detection, lane detection is a well- 
researched area of computer vision with applications in 
autonomous vehicles and driver support systems. Despite 
perceived simplicity of finding white markings on a dark 
road, it can be very difficult to determine lane markings on 
various types of road. These difficulties arise from shadows, 
changes in the road surfaces itself, and differing types of lane 
markings. A lane detection system must be able to pick out 
all manner of markings from cluttered roadways and filter 
them to produce a reliable estimate of the vehicle position 
and trajectory relative to the lane as well as the parameters of 
the lane itself such as its curvature and width. 
Existing methods for lane marking detection are usually 
based on edge detection (McDonald, 2001) and gradient 
analysis (Lu, 2007). Use of edges makes detection results 
sensitive to noise, changes in lighting conditions and 
shadows. Another approach uses steerable filters (McCall, 
2004) which can be convolved with the input image and 
provide features that allow them to be used to detect both 
dots and solid lines while providing robustness to cluttering 
and lighting changes. 
As long as these methods were designed for autonomous 
vehicles, they aim at tracking of lane marking in video. In our 
work the goal is to detect lane marking in still images of road 
surface. Also our task is to detect precise contours of lane 
marking instead of just determining lane marking direction. 
This task is closely related to the field of semantic image 
segmentation, therefore the method we propose for detection 
is based on semantic segmentation of rectified road images. 
Rectified images can differ substantially depending of 
roadway material, time of survey and weather conditions. 
Therefore automatic detection tuned on one road image can 
perform poorly on other images. For this reason we have 
developed a detection algorithm which is automatically tuned 
with the aid of user interaction in order to perform best on 
each particular road. This allows accounting for specific 
characteristics of every particular road, or even a road 
section.