1. INSTRUCTIONS 
1.1 Instructions 
Airborne Light Detection and Ranging technology has enjoyed 
rapid development in the photogrammetry and remote sensing 
community in recent years. Though many applications of 
LiDAR data have been carried out in fields such as topographic 
mapping, forestry parameter retrieval(Hyypp, 2000)(Andersen, 
2007;Melzer, 2004), power-line detection(Feng, 2009; Melzer, 
2004;Xu, 2008), 3D urban modeling(Gamba, 2000;Jic, 
2006;Rottensteiner, 2002;Sampath, 2007;Habib, 2009), and true 
orthophoto production(Kim, 2006;Riao, 2007), to mention only 
a few. However, it is a pity that LIDAR data lack of spectral 
information due to the monochromatic property a laser 
transmitter adopted(Baltsavias, 1999), no matter what kinds of 
laser sources used. On the other hand, imageries acquired by 
conventional air-or-space borne sensors are imaged within 
visible or near-infrared band of spectrum, therefore, full of 
semantic information. To combine their respective advantages 
of LiDAR data and imagery can not only provide extra 
information for thematic mapping, it is also the requirement of 
ortho-photo production, in which accurate Digital Surface 
Model is a must. 
Co-registration of airborne laser scanning data and imagery is 
the first step when combining the usages of the two datasets are 
considered. Since 3D property in nature, laser scanning datasets 
are difficult to register with imagery by conventional image-to- 
image registration methods. Though there is lot of literature 
describes image-to-image registration(Brown, 1992), both 
automatically or semi-automatically, however, little literature 
concerns the problem of registration between LiDAR data and 
imagery up to date. In some research straight line 
features(Habib, 2005) and planer patches(Kwak, 2006;Bang, 
Habib 2008) were used in two separate methodologies as the 
primitive of choice for the co-registration of the 
photogrammetric datasets to the LiDAR coordinate system. 
The approach using straight line features and planer patches 
starts with generating a photogrammetric model through a 
photogrammetric triangulation using an arbitrary datum without 
knowledge of any control information(Habib, 2005). To 
incorporate photogrammetric straight line in the registration 
model, the end points of “tie line” have to be identified in one 
or more images, providing for collinearity equations. 
Intermediate points are measured on this line in all images 
where it appears. For each intermediate points, a coplanarity 
constraints is used. This constrain states that the vector from the 
perspective center to any intermediate point along the line is 
contained within the plane defined by the perspective center of 
that image and the two points defining the straight line in the 
object space. Similar to the case of the line features, on the 
characteristics of planar patches in both datasets. The core 
principle behind this methodology is that in the absence of 
systematic error, LiDAR points belonging to a certain planar- 
surface should coincide with the photogrammetric path 
representing the same object space surface. In other words, the 
volume of the pyramid with its vertex at the LiDAR point and 
its base at the corresponding photogrammetric patch should 
equal to zero. Though good results achieved, Harbib's method 
has shortages in term of the following aspects: a) only multiple 
frames with adequate overlapped regions can be registered, b) in 
their model, traditional photogrammetric workflow is used to 
obtain exterior elements, which is a two-step procedure consists 
of relative and absolute orientation. Errors can be accumulated 
in the procedure, therefore, affect the final registration accuracy, 
c) their procedure is complex and is not a computational cost- 
effective one. 
Another factor must be considered when registering LiDAR 
data and imagery is the correspondence between image 
resolution and LiDAR data density. When the application of 
urban planning and management is concerned, resolution of 
photogrammetric images usually ranges from 5cm to several 
tens of centimeters according to the configuration of modern 
photogrammetric cameras, while point spacing of LiDAR point 
clouds ranges from 0.2 to 2 meters. What is the optimal 
resolution an image should be when it is registered to LiDAR 
data with given density or vice versa? We view this problem as 
scale analysis. To the authors best knowledge, there is no 
literature concerning this problem. 
In summary, the existing problems of registration of LiDAR 
point clouds data and remote sensing images are as following: 
registration process is complex, requiring to complete in two 
steps. First, three-dimensional relative orientation is used to 
generate image corresponding point clouds; second, 
corresponding point clouds is taken to register with LiDAR 
point clouds; (2) mathematic formulation of registration 
primitives is not concise enough; (3) lack of registration method 
which is suitable for single frame image and multiple frame 
images with LiDAR point clouds simutanuously; (4) there is 
lack of scale analysis. 
2. THE SELECTION AND EXPRESSION OF 
REGISTRATION PRIMITIVES 
2.1 The Selection of Registration Primitives 
Though much different in nature concerning the LiDAR data 
and imagery, the registration of them can also be decomposed 
into four essential problems: the extraction of registration 
primitives, the establishment of similarity measurement, the 
selection of transformation function and the strategy for 
matching(Brown, 1992). 
Registration primitives should satisfy the remote sensing data’s 
characteristics that are obvious, even distribution and easy to 
extract, and it cannot be affected by different sensor data's 
geometrical or radiation deformation, noise, scene changes and 
other factors. The primitives which can be used for registration 
in current literature mainly are areas, such as forest, lake and 
champaign (Zitova, 2003,Flusser, 1994, Goshtasby, 1986) and 
points, such as angular point of areas, point of lines intersection 
(Stockman, 1982, ) and inflection point with large radian(Ali, 
1998), as well as linear features, including ground objects’ 
edge(Dai, 1997) and split surfaces' intersection lines. 
This paper regards linear feature as registration primitives, 
mainly because LiDAR is characteristic of discreteness, which 
cannot accurately choose control points. Moreover, linear 
feature has the following advantages: 
€  Lincar fcature has a certain degree of scalability. Any two 
points of linear feature can state the whole linear feature; 
€ In 2D and 3D data’s registration transformation model, 
linear feature has more powerful constrained condition 
than plane feature; 
€ The expression ways of linear feature are various, which 
can be parametric representation or expressed by any two 
misalignment points in straight line; 
® Any point in straight line can be expressed b 
parameters. 3 
Unlike image data, the selection of tie points from LiDAR 
dataset for registration is almost impossible, since LiDAR data 
are so called point clouds, what are the discrete return echoes 
representing the 3D geographic coordinate values. Therefore, it 
is of great difficulty to precisely register LIDAR data and image 
if we use tic points as the registration primitives. Though 
patches are usually good candidates as registration primitives, 
y line 
382 
  
the extractic 
nor from p 
registration 
22 The E: 
In our proc 
primitives ii 
and conside 
we propose 
many sidew 
most of tim 
degree. So r 
the building 
Differential 
extract prec; 
that the der 
vicinity of e 
  
Figure 1: ex 
The details « 
(D Fi 
(2) Se 
Fig 
dir 
ge 
(6) 
(6) So 
Wi 
the 
ele