The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part Bl. Beijing 2008 
1232 
is available. Over this data link the analogue video signal as 
well as the flight control data consisting of platform attitude, 
position, velocity, and time stamp data can be transmitted. The 
analogue transmission of the video stream has the advantage 
that the time delay of the actual data transmission can be 
neglected, in contrast to the delay in digital transmissions which 
is introduced by the data encoding and decoding processes and 
which varies depending on the contents of the video imagery to 
be encoded. The portable ground control station of the md4-200 
platform provides a data link receiver, which continuously 
writes the flight control data stream to a serial RS-232 interface 
and the video stream to a composite video signal output. 
Figure 5: System architecture and data processing chain of the video imagery integration 
In order to perform an accurate video geo-registration later in 
the processing chain, we require a time-coded video stream 
based on the same time reference as that of the available flight 
attitude data. In the prototype system this task is performed by a 
time code integration component, composed of a GPS receiver 
and a VITC (Vertical Interval Time Code) time code integrator. 
This time code integrator is synchronised based on the available 
PPS (pulse per second) signal of the GPS receiver. For each 
frame the absolute GPS time or rather the derived UTC is 
integrated. Because of the limited payload weight on the UAV 
platform we are forced to outsource this process to the ground 
control station. Subsequently, the analogue video signal is 
converted from analogue to digital and further processed with 
the developed video processing software components on the 
portable computer, which complements the ground control 
infrastructure. These software components enabling the video 
imagery integration are described in detail in section 4.2. 
In order to realise a real time processing solution, the available 
flight control data stream has to be connected to this computer. 
For subsequent offline video imagery integration, the time- 
coded video stream as well as the flight control data stream can 
be stored on a local drive (see Figure 5). 
4.2 Video processing 
The subsequent processing of the time-coded video stream and 
of the available flight attitude data happens with software based 
video processing filters. These software components have been 
implemented based on the Microsoft DirectShow framework. 
This framework provides the basic functionality for designing a 
customised video processing application on Microsoft platforms. 
Different filters which implement functionality such as the 
reading and writing of streams, video conversion, video 
rendering or video multi- and de-multiplexing are available. For 
more details on DirectShow interested readers are referred to 
(Pesce, 2003). To realise a custom-built video processing 
application these filters can be freely combined to a filter graph. 
When this filter graph is executed, each frame of the video 
stream is processed serially by each filter defined in the graph. 
For the video geo-registration and the integration into virtual 
globes different new filters have been implemented, which are 
described in the following two sections. Figure 6 presents the 
simplified filter graph with the newly developed filters. 
4.2.1 Video imagery geo-registration 
As depicted in Figure 6 the geo-registration process consists of 
the VITC time code reader and the flight data integrator filters. 
The first filter extracts the integrated time code and makes the 
time stamp of each frame available to the following filters. The 
second filter implements the actual geo-referencing process. In 
this process the unknown sensor model, which consists of the 
exterior and interior orientation of each frame is estimated. For 
this estimation the direct geo-referencing approach is 
implemented. The direct georeferencing approach is introduced, 
for example, in (Mostafa and Hutton, 2005), (Cramer, 2001) or 
in (Skaloud, 1999), who give a good overview of characteristics 
and problems. In contrast to direct geo-referencing applications 
in airborne photogrammetry (cp. Cramer et al., 2000), we have 
to rely on flight attitude states from low quality INS/GPS 
sensors only. With the aid of the time stamp of each video 
frame, the synchronisation between the current video frame and 
the flight attitude data can be established. As shown in Figure 6