! 
* Corresponding author. 
In: Paparoditis N., Pierrot-Deseilligny M„ Mallet C.. Tournaire O. (Eds). 1APRS. Vol. XXXVIII. Part ЗА - Saint-Mandé, France. September 1-3. 2010 
AUTOMATIC DETECTION OF BUILDINGS WITH RECTANGULAR FLAT ROOFS 
FROM MULTI-VIEW OBLIQUE IMAGERY 
J. Xiao *, M. Gerke, G. Vossehnan 
Faculty of Geo-Information Science and Earth Observation. University of Twente, 75 MAE Enschede, The 
Netherlands - (jing, gerke, vosselman)@itc.nl 
Commission III. WG 111/4 
KEY WORDS: Oblique Image. Multiple views, Building Detection, Plane Sweeping, Cross Correlation 
ABSTRACT: 
Automatic building detection plays an important role in many applications. Traditionally, buildings were detected from monocular 
or multi-view images, DEM or DSM. and laser scanner data. Oblique imagery is a relatively new' data source with distinct 
advantages: it provides detailed information on facades and multiple views from various perspectives. In this paper we present 
exploration tests using only oblique imagery for the detection of buildings, mainly focus on rectangular flat roof type. It has two 
major stages: 1) generating robust facades from 3D lines extracted from multiple images; and 2) determining the height and roof 
outlines from a single facade of one building by plane sweeping and image segmentation. All tested buildings w'ith rectangular flat 
roofs w'ere successfully distinguished from other buildings, and their height and roof outlines were correctly detected. The novelty of 
this paper is the combination of geometric and radiometric approaches requiring only oblique imagery. This approach can be 
improved by adjusting the segmentation method and adding parameters for plane sweeping in order to be successful on buildings of 
other types. 
1. INTRODUCTION 
Automatic building detection is important in many applications, 
for instance map updating, city modelling and urban planning. 
Various data sources have been used for building detection, 
including single or multiple overlapped airborne images (Müller 
and Zaum, 2005; Karantzalos and Paragios, 2009), DSM or 
DEM (Ma, 2005; Lu et al., 2006), InSAR (Thiele et al„ 2007) 
or laser scanning data (Oude Elberink and Vosselman, 2009), as 
well as combinations of these (Khoshelham et al., 2010). 
However, all these data sources provide only vertical scene 
information, thus making it difficult to distinguish single 
buildings when their roofs are connected and of homogenous 
appearance. 
Nowadays, oblique images w'ith large tilt angles are available. 
(Petrie and Walker. 2007). These provide abundant information 
on building facades. Objects are imaged w'ith stereo overlap 
from multiple directions. This imagery also brings some 
challenges not present in the use of ortho-images. The first is 
the variable-scale geometry caused by the tilt angle 
(Grenzdörffer et al., 2008). Another is occlusion objects can be 
self-occluded on one or two sides, or be occluded by higher 
nearby objects. 
The objective of this research is to develop a method to detect 
buildings from multi-view oblique images alone, without any 
other data sources. In this paper we only focus on buildings 
with flat roofs. The research questions are: 1) how to generate 
robust facades from line detection in oblique imagery: and 2) 
how to determine building height and roof outline in 3D. Some 
related work is firstly reviewed in section 2. 
2. RELATED WORK 
Multiple overlapped nadir images have long been popular for 
solving the problem of building detection (Roux and McKeown, 
1994; Baillard et al., 1999; Kim and Nevatia, 2004), but off- 
nadir imagery has also been applied in a few studies. For 
example, buildings were detected using shadow and w'all 
evidence in one oblique view (Lin and Nevatia, 1995), and 
large buildings were recognized from natural images 
(Malobabic et al., 2005). Terrestrial video is another source of 
image sequence for building detection (Tian et al., 2009). 
Oblique imagery combines the advantages of multiple over 
lapped view's and terrestrial imagery, but its applications are not 
so wide till now'. The main application is texture extraction for 
3D modelling (Frueh et al., 2004; Wang et al., 2008). A few 
studies have been carried out on making use of the inherent 3D 
information for dense matching (Le Besnerais et al., 2008; 
Gerke, 2009). Another study validating road data from imagery 
(Mishra et al., 2008). The accuracy of measurements from 
single-picture in a certain software using oblique imagery was 
assessed by Sukup et al. (2009). 
METHODOLOGY 
The methodology consists of two stages, each with sub-stages 
(Figure 1). In the first, the vertical facades of the buildings are 
generated from extracted 3D lines, which are considered robust 
but do not include all the lines in the scene. In the second stage, 
heights and outlines of the roofs are detected by plane sweeping 
and segmentation from the vertical facades. 
251