ul 2004 
results 
sure 4). 
sults of 
1e final 
\ single 
  
  
e frame 
special | 
  
  
  
  
ingly. 
A| 
DE 
1 
Li 
JC 
HNO 
1e linear 
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B4. Istanbul 2004 
  
combination of the polygon vertices are then in the range 0 to 
| as required. 
3.3 Detection of Self-Occlusions 
The extraction of facade textures from photographs always 
leads to the problem that parts of the facades are not visible 
because of self-occlusions of the building. If no special care 
is taken, then erroneous pixel values are extracted for the 
occluded parts of the facade (see Figure 6 to Figure 8). To 
avoid such artefacts, invalid pixels that belong to other 
polygons must be identified and marked. 
Pixel-wise occlusion detection is realised in this approach by 
using the depth buffer algorithm. First, the depth value of the 
closest polygon is determined for each pixel in the 
photograph and stored in a depth texture. This can simply be 
done by rendering all polygons with the hardware depth 
buffer functionality enabled and by copying the resulting 
depth buffer into a 32 bit floating-point texture. A more 
efficient approach is to calculate the depth value in a pixel 
shader and render directly into the depth texture. Modern 3D 
graphics processors support the floating-point texture formats 
even as render targets. 
During texture extraction, the depth value is read out in the 
pixel shader using the same texture coordinates as for the 
colour lookup. After the perspective divide is applied to the 
texture coordinates, the z-component holds the depth value 
for the current polygon. A comparison of these two depth 
values then determines if the pixel in the colour texture 
belongs to the polygon. If c.g. the value from the depth 
texture is lower then the computed value, then the polygon is 
occluded at this pixel by another polygon. Figure 9 shows 
some results where occluded pixel values have been 
blackened out. To suppress artefacts caused by precision 
errors, the depth test is done by applying a small depth bias 
in the depth test. 
3.4 Image Fusion 
As only parts of a building or even of a fagade are typically 
captured in one single image, the colour information from 
several photographs that were taken from various positions 
need to be combined to generate the final facade textures. 
One simple approach is to extract several textures for the 
same polygon from all available photographs and then use 
the one with the fewest pixels marked as occluded (see e.g. 
Figure 10). Other criteria may possibly be the photograph 
taken closest to the facade or the one with the best viewing 
angle. 
The problem of image fusion done in the hardware is how to 
get the graphics pipeline to decide from which image a pixel 
should be taken. The solution is to process all images and 
make the hardware accept or reject pixel values by using the 
depth, stencil or alpha test. Even though the approach is brute 
force, it is still very efficient with the hardware support. 
The presented per pixel approach merges the final facade 
texture by using the colour value of only the closest, non- 
occluded pixel found in all images. The occlusion detection 
Works as described in the previous section, but now with the 
depth test enabled on the hardware. The output of the pixel 
shader that is used for the hardware depth buffer test is the 
calculated depth value for non-occluded pixels and 1.0 (the 
farthest possible depth value) for occluded pixels. Because it 
is usually better to have a wrong colour value rather then no 
  
Figure 6. Input photograph showing Stuttgart State Theatre. 
Bia 
ite 
  
2 
  
Figure 8. Building model automatically textured without 
occlusion culling. 
  
Figure 9. Building model automatically textured with 
occlusion culling. The black pixcls were marked 
as occluded texture pixels.