International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B7, 2012 
XXII ISPRS Congress, 25 August — 01 September 2012, Melbourne, Australia 
5. GEOMETRIC DATA FUSION METHODS 
5.1 Classification vectors 
Classification algorithms result in the assignment of pixels to 
different material classes, which are represented by 2D image 
space vectors in remote sensing software. Using the EOPs of 
the hyperspectral image, the reverse collinearity condition 
can be used to form rays for each point of a 2D line. As the 
image has no depth information, the lidar mesh is intersected 
by the rays to give points in object space, thus transforming 
the classification boundaries into 3D polylines and polygons. 
This 3D information can be used to extract real-world 
dimensions, allowing investigation of spatial relationships of 
material distribution. 
5.2 Point cloud classification and modelling 
The 3D classification boundaries obtained in Section 5.1 can 
be combined with segmented point clouds (Section 4.1) to 
create triangulated meshes for individual material bodies. 
This allows accurate calculation of surface area for the 
bodies, and estimation of volumes where appropriate 3D 
exposure is present. The procedure operates on a single 
material class at a time; i.e. a set of lines and a segmented 
point cloud representing one class. Each line is processed in 
turn, first checking whether it is the outer boundary of a 
body, or whether it is an inner hole indicative of a different 
material class. All points of the point cloud are projected to 
image space using the point of view of the capturing image, 
to test whether they are inside the polygon. Polygons with no 
containing points are designated as holes. Still working in 
image space, the points and lines are triangulated, using the 
holes as constraints for triangle removal. Finally, the internal 
vertices of the triangulated body are assigned their original 
3D lidar positions, and the points on the line segments are 
projected to 3D as in Section 5.1 (Fig. 6). 
  
Figure 6. 3D material bodies created using triangulated 
points and classification boundaries, and 
superimposed on textured model (Garley 
Canyon). Inset shows detail of points and lines. 
5.3 Assignment of geometric properties to imagery 
Using the TLS geometry allows hyperspectral image 
information to be accessed in 3D in object space units. For 
remote sensing processing it can be useful to have per-pixel 
geometric information, such as for topographic correction 
and spatial analysis of results. For each image pixel, a ray is 
defined and intersected with the 3D mesh, as in Section 5.1. 
545 
For pixels with no 3D data, such as those representing sky or 
background, a no-data value is assigned. The surface normal 
at the intersection point can be used to calculate slope and 
aspect values for each pixel (e.g. Fig. 7), which are useful for 
topographic correction and geological analysis. In addition, 
the range from the camera centre to the intersected points can 
be used for calculating object sample distance, giving a quick 
approximation for material class areas in the remote sensing 
software environment. 
  
Figure 7. Slope-encoded image that can be used as a 
processing mask in remote sensing software. 
White areas are outside of the 3D model. Garley 
Canyon image used (Fig. 2). 
5.4 Accuracy considerations 
Multi-sensor fusion products rely on accurate co-registration 
of the component techniques to be used appropriately. For 
close range hyperspectral data, the image registration is 
critical and should be inspected. Whilst statistical confidence 
is a result of bundle adjustment, it is also important to 
evaluate the accuracy in an integrated way. Photorealistic 
models (Section 4.2) textured with conventional photos and a 
hyperspectral product give a qualitative indication of 
registration accuracy, as the user can blend between the 
layers to see the proximity of conjugate areas in both data 
types. Projection of the classification vectors to 3D allows 
their position with respect to the photorealistic model to be 
compared geometrically (Kurz et al., 2011). 
6. CONCLUSIONS 
This paper has outlined the potential of integrated TLS and 
close range hyperspectral data for simultaneously analysing 
the geometry and distribution of materials. Examples from an 
application in geology have been used to illustrate the data 
fusion products, though the methods have potential use in 
many other disciplines. The two data types are highly 
complementary, and should be used together both during 
processing and for later analysis. Results are often visual, and 
it may be a challenge to link all information in a single 
environment for interpretation. Photorealistic modelling with 
multiple textures allows the hyperspectral results to be 
spatially related to higher resolution conventional photos, 
and lidar geometry, in a single interactive viewing 
framework. Linking 2D image data to 3D introduces 
quantitative means for analysing classification results, and 
allows co-registration accuracy to be inspected both 
statistically and visually. 
ACKNOWLEDGEMENTS 
This work was supported in part by the Research Council of 
Norway's Petromaks programme (grants 163264 and 
176132). Statoil ASA is thanked for supporting fieldwork at