ISPRS Commission II, Vol.34, Part 3A „Photogrammetric Computer Vision“, Graz, 2002 
are generated based on a spatial model of the visible 
environment. In our application this data is provided by the 3D 
city model. Of course the virtual computer graphic objects have 
to be correctly overlaid to their corresponding primitives in the 
real world as they are observed by the user. For this reason the 
accurate tracking of the actual position and orientation of the 
user in order to enable a precise mapping of the data is 
required. 
Within an urban environment, AR can for example be applied 
for the presentation of name labels or additional alphanumeric 
data appearing to be attached to a side of a building. In addition 
to the visualization of these virtual signposts, more specialized 
applications could aim on the display of information based on 
“X-ray vision” in order to present features normally not visible 
for the user. Typical objects of interest are features hidden 
behind the facades of a building like the location of rooms or 
information on infrastructure like the position of power-lines. 
The integration of augmented reality into a tourist information 
system is another application for this kind of technique. As an 
example, for the old town of Heidelberg a mobile tourist 
information system has been developed (Malaka & Zipf 2000). 
Within this system preliminary to his actual visit a potential 
tourist can virtually walk through the 3D city model to allow 
the planning of real tours. On-site, the visible environment is 
enriched by information relevant for each building. By these 
means queries on thematic information like opening hours of 
museums or the generation and overlay of historic views can be 
realized. À similar system, helping a user to navigate through a 
build-up area is also described by (Hôllerer et al. 1999). Using 
a head mounted display, the names of buildings are presented to 
the user depending on his actual field of view. Additionally, by 
pointing to the buildings supplementary information is made 
accessible via an integrated wireless access to the internet. 
This so-called telepointing feature is also realized within our 
research project NEXUS (Fritsch et al. 2000), which is the basis 
of the work presented within this paper. For simplification of 
the overall system, in this application the head-mounted display 
is replaced by an image of the user's environment. This image 
can for example be captured by a camera integrated into a small 
hand-held display. 
a 
7) 
Sue EE 
stasihgus 
Ticketservice 
Opernhaus 
   
  
el 
  
Figure 1: Prototypical telepointing application 
An exemplary application based on our current prototype is 
depicted in Figure 1. Simultaneously to the capture of the 
image, the position and orientation of the camera is determined 
by a DGPS receiver and a digital compass. This information is 
sufficient to calculate the viewing frustrum for the captured 
image, which then can be projected to an ortho image or a map. 
By pointing to a specific object of interest in the image, 
corresponding object related information as it is for example 
provided by a website is presented by the graphical user 
interface. These websites then give access to services like ticket 
sales if for example a theatre is visible. Currently, the system is 
realized within a standard GIS software package. In the final 
system the NEXUS platform will provide both the management 
of the positioning components and the provision of the spatial 
models. A small mobile device (PDA) will be utilized as 
personal NEXUS station and information between platform and 
station will be exchanged by wireless communication. 
3. INTEGRATED ORIENTATION OF TERRESTRIAL 
IMAGES 
In order to enable a precise mapping of the 3D building model 
to the captured image, an accurate tracking of the imaging 
device is required. Based on this information the access to the 
augmented world data, i.e. the spatial model of the user's 
environment enriched by additional objects is feasible. This can 
then be realized by pointing to respective regions of interest 
directly on the image display. As it it realised in our system the 
user's actual position can be provided by the use of a small 
DGPS receiver and a digital compass in outdoor areas. 
Nevertheless, in situations where the provided accuracy is not 
sufficient, the already captured image can additionally be used 
for a further improvement of image georeferencing. 
3.1 Directly Measured Exterior Orientation 
The accuracy of the exterior orientation as provided by the 
available DGPS receiver could be verified to several meters for 
the positional accuracy, whereas the orientation accuracy as 
provided by the digital compass and a tilt sensor resulted in an 
error of 1° — 2°. Figure 2 depicts the application of these low- 
cost components for the selection of the visible building as well 
as the initial transformation of the building's wire-frame to the 
directly geoereferenced terrestrial image. 
  
Figure 2: Projection of building based on DGPS and digital 
compass measurement 
Even though this coarse mapping is sufficient for some 
applications, it has to be refined if highly localized information 
has to be presented to the user. For this purpose an automated 
appearance-based detection of buildings in terrestrial images is 
applied. The problem is stated as follows: From an image with 
a given approximated exterior orientation and a three- 
dimensional CAD Model of the building, detect the exact 
location of the building in the image and use this information in 
A - 139