Petra Zimmermann 
  
A NEW FRAMEWORK FOR AUTOMATIC BUILDING DETECTION ANALYSING MULTIPLE 
CUE DATA 
Petra ZIMMERMANN 
Swiss Federal Institute of Technology, Zurich, Switzerland 
Institute for Geodesy and Photogrammetry 
petra@ geod.baug.ethz.ch 
  
KEY WORDS: Buildings, Feature Extraction, Object-oriented, Recognition 
ABSTRACT 
This paper describes a new framework for building recognition in aerial images. The proposed framework integrates 
several low-level image-processing algorithms to derive from different cues as colour, texture, edges and colour edges 
elevation data features to detect and recognise buildings. We developed an object-oriented Java based system for scene 
interpretation and reconstruction. Our system provides a set of classes for feature extraction, data storage, reasoning and 
visualisation, including user interaction. The core is an easy-to-expand repository of basic features and algorithms. Each 
step contains a possible exchange between data and processing in 2D and 3D to improve results on these levels. Due to 
its implementation in Java there is no restriction for future distributed applications over networks. We present a system 
to derive spatial data in 2D and 3D, based on independent components, knowledge and control able to distinguish 
between man-made objects like roads, buildings, bridges and places and natural objects like trees, vegetation on the 
ground. Results of building detection in a new data set from Zürich are presented. 
1 INTRODUCTION 
The number and variety of applications based on 3D data of terrain with or without buildings increases rapidly, since 
the necessary hardware becomes more powerful, costs more reasonable and visualisation faster. Many efforts in the area 
of building detection and reconstruction (Grün et al., 1999, Gülch et al. 1998, Henricsson, 1996) supported this 
development. Due to conditions in the particular application, terrain models and city models in 3D have to fulfil 
different requirements concerning resolution and accuracy. Building reconstruction and visualisation in 3D requires the 
previous careful detection of buildings. Present systems detect and reconstruct buildings in manifold ways, either 
through manual measurement, semi-automatic methods or for well defined situations automatically (Grün et al. 97). 
1.1 AMOBE I and AMOBE II 
After the successful completion of the first phase of the AMOBE project at ETH Zurich (Henricsson, 1996), focusing 
on reconstruction of buildings from multiple-view aerial imagery based on edges and planar patches attributed by 
colour information, we are now involved in a follow-up project AMOBE II, concentrating on remaining problems. 
AMOBE II is extended by building detection by integrating knowledge, basic models and multiple cues like colour, 
texture, colour edges, shadow and elevation data. AMOBE II will also improve building reconstruction (Scholze et al. 
2000) by integrating these cues, treat problems caused by occlusions and focus on complex scenes and building details. 
In this paper we present feature-based, region-and boundary-based image segmentation based on the cues colour, DSM, 
texture combined with edges applied to building detection. 
The first phase of AMOBE I proved how useful the strategy to process early in 3D instead of processing in 2D could be, 
and also the need for more extensive use of different cues became visible (Henricsson, 1998). In contrary to semi- 
automatic systems (Grün et al. 1999) where an operator marks buildings, or other systems that know about building 
locations through databases or maps (Niederóst et al. 2000), the AMOBE II system is designed to be able to detect 
automatically buildings and derive useful information to support reconstruction. Not only for building detection also for 
building reconstruction multiple cues and data sources are incorporated to improve the results (Park et al., 2000), 
(Zhang et al., 2000). 
We present our concept of a "toolbox" of algorithms for different cues and two examples of tools for building 
recognition in detail, a short description of the single cues and the first results with our new dataset. 
  
International Archives of Photogrammetry and Remote Sensing. Vol.. XXXIII, Part B3. Amsterdam 2000. 1063