B3, 2012 
n be obtained 
ased method to 
ure for LiDAR 
cy. 
area are plane 
an be regarded 
imitive used in 
framework and 
t 6 parameters 
mitive. Further 
ive is placed in 
  
'd 3D building 
the order of 
ical imagery / 
5 
xtract features 
  
  
    
    
Corners of 
buildings 
  
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B3, 2012 
XXII ISPRS Congress, 25 August — 01 September 2012, Melbourne, Australia 
Figure 2. Flowchart of the proposed method 
1. Recognize primitives and measure initial 
parameters. With the help of optical imagery and 
LiDAR point cloud, the building is decomposed into 
several primitives. Then the primitive’s parameters 
are measured roughly on LiDAR point cloud and 
optical imagery, such as length, width, height, 
orientation and translation of the primitive. These 
measurements can be used as fixed values 
(constraints) or initial values in the following 
optimization procedure. 
2. Extract features. Corners are detected/selected on 
the optical imagery, and planes are detected/selected 
in the LiDAR point cloud. These features will be 
used as observed values/observations in the following 
optimization procedure. 
3. Compute features. Based on the type and 
parameters of primitives, the 3D coordinates of the 
primitives’ features, such as corners, can be 
calculated. They will be used as model/computed 
values in the following optimization procedure. 
4. Optimize parameters. When a 3D building model 
has correct shape and is located in the correct place 
in 3D space, two conditions will be satisfied. Firstly, 
the  back-projections of  primitive's vertexes 
(computed features) on the optical image should 
perfectly superpose on the measured corners 
(extracted features). Secondly, the  primitive's 
vertexes should be exactly on the planes which are 
formed by LiDAR point cloud. These two conditions 
can be expressed respectively by Collinearity 
Equation and 3D Plane Equation, and then a cost 
function can be established using these two 
mathematical models. The inputs of this cost function 
are observed values, model values, and initial values 
above. When the optimization procedure is finished, 
the optimized/refined primitives’ parameters will be 
outputted. 
Finally, 3D building can be represented by these primitives 
with the optimized parameters. 
The proposed method was applied to a dataset of an ISPRS test 
project. The organizer of this test project evaluated the 
submitted reconstructed 3D model using reference data. In the 
next section, first is the description of test data, followed by 
the introduction of data processing, finally evaluating result is 
analyzed and discussed. 
3. EXPERIMENTAL RESULT AND DISCUSSION 
3.1 Description of Data Set 
The test data set was captured over Vaihingen in Germany. 
The data set is a subset of the data used for the test of digital 
aerial cameras carried out by the German Association of 
Photogrammetry and Remote Sensing (DGPF) (Cramer, 2010). 
The ground resolution of the digital aerial images is 8 cm. The 
Vaihingen test data set provided by DGPF also contains 
Airborne Laserscanner (ALS) data. The entire DGPF data set 
consists of 10 ALS strips. Inside an individual strip the average 
point density is 4 pts/m? (Haala et al., 2010). 
The test data consists of three test areas for which reference 
data for various object classes are available (Spreckels et al., 
2010). In this paper, Area 3 “Residential Area” was selected; it 
is a purely residential area with small detached houses. Most 
of buildings in this area can be represented by hip-roof 
primitive. Fig. 3 shows the digital image of this test area. 
  
Figure 3. Digital image of the test area 
3.2 Task and Data Processing 
This ISPRS Test Project has two tasks, Urban Classification 
and 3D Building Reconstruction. The task of this paper is the 
latter. The goal of this task is to derive a complete, correct, and 
accurate segmentation of the roof planes in the provided data. 
The detailed 3D models of the building roofs in the test areas 
should be generated. The level of detail should correspond to 
LoD2 of the CityGML standard. 
The workflow of Fig. 2 was applied to the test data to generate 
3D building models. It should be noted, at current stage, some 
works were done in interactive mode. Both building's corners 
in images and building's planes in point cloud were manually 
extracted. 
3.3 Experimental Result 
After data processing, 3D building models were reconstructed. 
The requirement of submitted result of ISPRS Test Project is