DXF files containing closed 3D polygons corresponding to the 
boundaries of the reconstructed roof planes in the object 
coordinate system given by the respective test area. Although 
the proposed method generated 3D building models, only 
building roofs were included in submitted DXF file and shown 
in Fig. 4. 
Figure 4. Reconstructed 3D building roofs 
Both initial models and optimized models were back-projected 
to the image as wireframes, which are shown in Fig. 5. Blue 
wireframes are back-projections of initial models and green 
wireframes are back-projections of optimized models. It shows 
an advantage of our method. Although some initial models 
obviously deviate from the true shapes and places, after 
optimization they can be corrected and fit the true buildings 
well. The comparison of whole test area is shown in Fig. 1. 
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B3, 2012 
XXII ISPRS Congress, 25 August — 01 September 2012, Melbourne, Australia 
  
   
    
   
    
     
  
  
  
  
  
  
  
  
  
  
  
      
  
Figure 5. Wireframes of initial models and optimized models 
3.4 Evaluating Result and Discussion 
The reference for Vaihingen was generated by 
photogrammetric plotting carried out by the SIRADEL 
company in France (www.siradel.com), following the 
guidelines used by RAG in Area 1 (Spreckels et al., 2010). The 
feedback of ISPRS Test Project is a text file containing the 
evaluation results. The evaluation consists of an analysis of the 
quality of the segmentation and an analysis of the geometrical 
errors of the submitted models compared with reference. 
Because currently primitive's types and initial parameters are 
not decided in an automated way, and the key feature of the 
proposed reconstruction method is the ability to compute 
optimized primitives’ parameters, so we concentrated more on 
geometrical accuracy. The evaluation of geometrical accuracy 
part of the report file is listed in Tab. 1. The geometrical error 
is evaluated by determining the RMS errors of building roof 
vertices (only for roof planes correctly segmented) and of an 
overall analysis of the height differences between the 
submitted models and the reference. 
  
Evaluation of Geometrical Accuracy: 
  
Distance threshold: 3.0 [m] 
  
Total RMS of extracted | 0.80 [m] (determined from 
  
  
  
boundaries: 840 of 913 possible F 
correspondences) 
Total RMS of centres of | 0.49 [m] / 0.56 [m] 
gravity of extracted objects (X | (determined from 109 of 133 
7 Y. possible correspondences) 
Total RMS of reference | 0.44 [m] 
boundaries: (determined from 505 of 816 
possible correspondences) 
Total RMS of centres of | 0.90 [m] / 0.92 [m] 
gravity of reference objects | (determined from 142 of 183 
CX / Y: possible correspondences) 
  
Height errors: 
  
Total RMS of height | 0.39 m 
differences between planes: 
  
  
RMS of height differences | 0.22 m 
between planes found to 
correspond: 
  
  
  
Table 1. Evaluation of geometrical accuracy in the report file 
And a few images that visualize these results are also provided. 
Two of these images are shown below. 
Fig. 6 is the evaluation of building detection on a per-pixel 
level. In this figure, yellow means correct roofs, and blue 
means missed roofs, and red areas is the background but 
reconstructed as roofs by mistake. 
Fig. 7 is difference between two DSMs which were derived 
from the roof planes of the result and the reference respectively. 
The difference is only evaluated for pixels where a plane was 
found in both data sets; all other pixels are displayed in white.