ul 2004 
fer set 
ition of 
ere, the 
(3) 
(4) 
(5) 
(6) 
f pixels 
rmula is 
unately, 
eraphics 
3D 9.0 
ntrol in 
ude this 
tion can 
The 2D 
1c image 
e. In the 
n vector 
ites. The : 
i colour 
; 
a vendor 
is should 
graphics 
ich also 
  
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B4. Istanbul 2004 
  
The calibration of the camera was done using the bundle 
program Australis (Fraser, 1997). Camera calibration files are 
automatically loaded and the correction textures are 
generated as needed. The input photographs were taken at 
resolution 1280 * 960 pixels, but resolutions up to 2048 * 
2048 are also supported. This limitation comes from the fact 
that textures in the graphics hardware are limited to this size. 
The output resolution of the facade textures is at this point 
fixed at 256 * 256 pixels. 
The performance analysis has been conducted on a standard 
PC with an Intel 4 3.0 GHz Processor, 1 GB of DDR-RAM 
and a graphics card that is based on the ATI 9800 GPU with 
256 MB of graphics memory. The test results are given in 
Table 1. It should be noted that 8 or 16 input photographs for 
per pixel image fusion is not practical. This rather high 
number was solely used to show the speed of the approach. 
Nevertheless, the extraction time with all features enable is 
still below one second. 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
time 
: Extraction Process model model B 
images 
A 
1 3] ms 47 ms 
1 Lens Correction 32 ms 62 ms 
1 Occlusion Detection 32 ms 63 ms 
1 Occlusion D. + Lens C. 46 ms 78 ms 
8 Image Fusion (per Pixel) 172 ms 328 ms 
8 [Image Fusion + Lens C. 203 ms 391 ms 
16 Image Fusion (per Pixel) | 375 ms | 813 ms 
16 Image Fusion + Lens C. 422 ms 829 ms 
Table 1. Extraction times measured for model A 
(Rosensteinmuseum, 71 polygons) and model B 
(Stuttgart State Theatre, 149 polygons). 
5. CONCLUSION AND FUTURE WORK 
This article described the concept and the implementation for 
hardware-based texture extraction of photo-realistic façade 
textures. The implementation of such a system is shown to be 
very simple by using standard 3D APls and shader 
languages. Fast extraction is possible on commodity PC 
hardware equipped with a 3D graphics processing unit and 
the resulting façade textures proved to be of very high 
quality. The resulting building models are automatically 
mapped by perspectively correct textures and can therefore 
be used for real-time visualisation. 
As the system has a low response time, it has the potential to 
be extended towards a semi-automatic tool, which allows the 
refinement of the model based on manual measurement in 
terrestrial images. The manual fitting of available building 
geometry to terrestrial images is often required due to 
remaining errors in the building model. Such errors are of 
nuisance when the correspondence between object and image 
is not exactly given and lead to artefacts or even wrong 
facade textures. Hardware-based texture extraction will allow 
a real-time visualisation of the textured 3D model, so that the 
operator can immediately observe the geometric changes. 
The future work will be to speed-up the overall process by 
doing some pre-processing of the geometry on the main 
CPU. Backface culling could e.g. be pre-computed for each 
image and stored in a backface table. Not all images would 
need to be processed for each polygon anymore. Another 
area of improvement is the quality for per pixel texture 
fusion. Alpha blending might help to reduce artefacts if parts 
of the texture can not be aligned correctly because of errors 
in the exterior orientation. As a combination of per-polygon 
and per-pixel image fusion promises the best results, adapted 
algorithms shall further be developed. 
In order to address occlusions by other objects, the presented 
system could be extended to a semi-automatic tool where the 
operator marks pixels or regions in the photograph as invalid. 
These pixels will not be used in the final texture, but rather 
colour values from other photographs are used or the missing 
pixel colours are reproduced by subsampling algorithms. 
6. ACKNOWLEDGEMENTS 
The research described in this paper is founded by "Deutsche 
Forschungsgemeinschaft" (DFG  — German Research 
Foundation). The research takes place within the Center of 
Excellence No. 627 *NEXUS — SPATIAL WORLD MODELS FOR 
MOBILE CONTEXT-AWARE APPLICATIONS" at University of 
Stuttgart. The geometry of the building models is provided 
by Stadtmessungsamt Stuttgart. 
7. REFERENCES 
Baltsavias, E. Grün, A. and van Gool, L., 2001. Automatic 
Extraction of Man-Made Objects from Aerial and Space 
Images (II). Swets & Zeitlinger B.V., Lisse, The 
Netherlands. 
Brown, D.C. 1971. Close-Range Camera Calibration. 
Photogrammetric Engineering, 37 (8). pp. 855-866. 
Fernando, R. and Kilgard, M., 2003. The Cg Tutorial. 
Addison-Wesley. 
Fraser, C.S., 1997. Digital Camera Self-Calibration. /SPRS 
Journal of Photogrammetry and Remote Sensing, Vol. 52, 
pp. 149-159. 
Früh, C. and Zakhor, A., 2003. Constructing 3D City Models 
by Merging Aerial and Ground Views. IEEE Computer 
Graphics and Applications, Vol. 23 No. 6, pp. 52-61. 
Gray, K., 2003. The Microsoft DirectX 9 Programmable 
Graphics Pipeline. Microsoft Press. 
Kada, M., Roettger, S., Weiss, K., Ertl, T. and Fritsch, D., 
2003. Real-Time Visualisation of Urban Landscapes Using 
Open-Source Software In: Proceedings of the ACRS 2003 
ISRS, 24^ Asian Conference on Remote Sensing & 2003 
International Symposium on Remote Sensing, Busan, Korea. 
(On CD-ROM) 
Klinec, D. and Fritsch, D., 2003. Towards Pedestrian 
Navigation and Orientation. In: Proceedings of the 7" South 
East Asian Survey Congress, SEASC’03, Hong Kong. (On 
CD-ROM) 
Microsoft, 2003. DirectX Documentation for C++. Microsoft 
DirectX 90 SDK. http://msdn.microsoft.com/library/ 
default.asp?url=/downloads/list/directx.asp 
Shreiner, D., Woo, M. and Neider, J., 2003. OpenGL 
Programming Guide (Version 1.4), Addison-Wesley.