3D CLASSIFICATION OF CROSSROADS FROM MULTIPLE AERIAL IMAGES USING 
MARKOV RANDOM FIELDS 
S. Kosov * *, F. Rottensteiner *, C. Heipke ^, J. Leitloff ® S. Hinz" 
* Institute of Photogrammetry and Geolnformation, Leibniz Universität Hannover, Germany - 
(kosov, rottensteiner, heipke)@ipi.uni-hannover.de 
^ Institute of Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology, Germany - 
(jens.leitloff, stefan.hinz)@kit.edu 
Commission III, ICWG III/VII 
KEY WORDS: Markov Random Fields, Contextual, Classification, Crossroads 
ABSTRACT: 
The precise classification and reconstruction of crossroads from multiple aerial images is a challenging problem in remote sensing. 
We apply the Markov Random Fields (MRF) approach to this problem, a probabilistic model that can be used to consider context in 
classification. A simple appearance-based model is combined with a probabilistic model of the co-occurrence of class label at 
neighbouring image sites to distinguish up to 14 different classes that are relevant for scenes containing crossroads. The parameters 
of these models are learnt from training data. We use multiple overlap aerial images to derive a digital surface model (DSM) and a 
true orthophoto without moving cars. From the DSM and the orthophoto we derive feature vectors that are used in the classification. 
One of the features is a car confidence value that is supposed to support the classification when the road surface is occluded by static 
cars. Our approach is evaluated on a dataset of airborne photos of an urban area by a comparison of the results to reference data. 
Whereas the method has problems in distinguishing classes having a similar appearance, it is shown to produce promising results if a 
reduced set of classes is considered, yielding an overall classification accuracy of 74.8%. 
1. INTRODUCTION 
The automatic detection and reconstruction of roads has been an 
important topic of research in Photogrammetry and Remote 
Sensing for several decades. Considerable progress has been 
made, but the problem has not been finally solved. The 
EuroSDR test on road extraction has shown that road extraction 
methods are mature and reliable under favourable conditions, in 
particular in rural areas, but they are far from being practically 
relevant in more challenging environments as they exist in 
urban or suburban areas (Mayer et al., 2006). 
One of the main reasons for failure of road extraction 
algorithms noted by (Mayer et al., 2006) is the existence of 
crossroads, due to the fact that model assumptions about roads 
(e.g., the existence of parallel edges delineating a road) are hurt 
there. For this reason, specific models for the extraction of 
crossroads from images have been developed. Barsi and Heipke 
(2003) used neuronal networks for a supervised per-pixel 
classification of greyscale orthophotos in order to detect areas 
corresponding to crossroads, combining radiometric and 
geometric features. However, only examples for rural areas were 
shown. Ravanbakhsh et al. (2008a, 2008b) used a model based 
on snakes to delineate outlines of road surfaces at crossroads, 
including the delineation of traffic islands. The main reasons for 
failure of that method were occlusion of the road surface by cars 
and a complex 3D geometry, e.g. at motorway interchanges. 
The problem of occlusion by cars could be overcome if the 
position of cars were known in the images. Extensive overviews 
about methods for vehicle detection from optical aerial imagery 
can be found in (Stilla et. al., 2004) and (Hinz et. al., 2006). 
  
* Corresponding author. 
In this paper we propose a new method for the classification of 
Scenes containing crossroads as a first step of a 3D 
reconstruction. Markov Random Fields (MRF; Geman & 
Geman, 1984) are employed for a raster-based classification. 
MRF offer probabilistic models for including context in the 
classification ^ process by considering the statistical 
dependencies between the class labels at neighbouring image 
sites; cf. (Li, 2009) for more details on MRF and their 
applications in image analysis. We use multiple-overlap aerial 
images in order to derive a Digital Surface Model (DSM) that is 
used in the classification process to make it more robust with 
respect to ambiguities of the appearance of objects in a 2D 
projection of the scenes. In addition, we include information 
about cars by integrating the output of a car detector into the 
process. Our method is evaluated using 55 crossroads of the 
Vaihingen data set of the German Society of Photogrammetry, 
Remote Sensing and Geoinformation (DGPF). 
2. MARKOV RANDOM FIELDS 
Markov random fields (MRF) provide probabilistic models of 
context for the image labelling problem (Geman & Geman, 
1984; Li, 2009). Given image data y consisting of N image sites 
i € S with observed data y;, 1.e., y = (y, y» … va", where S is 
the set of all sites, we want to assign a discrete class label x; 
from a given set of classes C to each site i. In this context, an 
image site can correspond to a single pixel or to an image 
segment. MRF are undirected graphical models that assume the 
data y; at image site / to depend on the class label x; at that site. 
In addition, the class label x; is modelled to be statistically 
  
   
  
  
   
  
  
  
  
   
  
  
  
  
  
  
  
  
  
  
  
    
  
   
   
   
   
   
   
   
   
   
  
   
   
   
   
   
  
  
  
  
   
   
   
   
   
   
   
   
   
   
  
    
ind 
ve 
cot 
lab 
po: 
dis 
OU] 
ve