2-5-6 
centerlines are obtained through a dynamic and global energy 
minimization procedure. Based on the shape from sequences 
scheme, a systematic combination of multiple constraints from a 
mobile mapping system can be implemented. 
Image sequence 
Figure 3. The concept of shape from image sequences 
This is a closely coupled bottom-up and top-down scheme. 
Firstly, an approximate 3-D road centerline model is set up by 
means of a B-spline approximation of the kinematic vehicle 
trajectory data. Secondly, the road model can be back-projected 
onto the image sequences using the known orientation 
parameters of the cameras. Guided by an established geometric 
and photometric image model of road centerlines, model-driven 
extraction of road centerline feature points is performed, 
constrained by the projected road centerline. Furthermore, a 
stereo-motion matching algorithm based on the stereo and 
motion constraints is applied to find a set of matched feature 
point pairs. After using photogrammetric triangulation of 
feature point pairs, a set of 3-D feature points of road 
centerlines in the object space is obtained. Thirdly, in order to 
refine the approximate road model using the information 
extracted from the image sequences, the model is defined as an 
active and deformable 3-D curve, the 3-D snake (Gruen and Li, 
1996; Gulch, 1996; Kass et al., 1988; Terzopoulos et al., 1988; 
Trinder, J., and H. Li, 1995). A physically-based deformation 
mechanism is incorporated such that the model can be 
progressively deformed driven by the internal and external 
energies. Internal energy arises from smoothness constraints 
representing the natural characteristic of the shape of road 
centerlines. It maintains the a priori knowledge about the shape 
of the road model. The 3-D feature points extracted from image 
sequences act as the external energy which enforces the model 
to deform towards its desired position. Under a combination of 
the actions of internal and external forces, the model will be 
deformed incrementally towards the final state at which forces 
from different sources are balanced. The newly deformed and 
refined model will then be used to update the approximate 
model, and the new information available from successive new 
images will be applied to further refine this model. The above 
process is executed repeatedly until the entire image sequence is 
processed. The finally obtained deformation curve can be 
treated as an optimized reconstruction result of 3-D road 
centerlines, because such a result is derived by using globally 
combined constraints. This method consists of three key 
components; 
• generation of an approximate 3-D shape model of road 
centerlines; 
• extraction of reliable information of road centerlines from 
image sequences; and 
• dynamic refinement of a physically-based 3-D road 
centerline model. 
Numerous test of real imagery demonstrate that this approach 
functions reliably even in situations where the road conditions 
are far from ideal or the imaging conditions are poor (Tao, 
1997). 
4.1.3 Feature Extraction and Object Recognition 
a. Overview 
In most cases, the two processes, feature extraction and object 
recognition, have to be combined or integrated in order to 
identify objects successfully. Without sufficient knowledge 
about objects, object recognition results can never be reliable. 
Some research work have been conducted to attempt to identify 
some specific objects, e.g., road signs, using image sequences. 
He and Novak (1992) have published their results on the 
detection of mile makers from mobile mapping images. The 
focus was placed on the recognition of numbers appearing on 
mile-maker images. Geiselmann and Hahn (1994) developed a 
strategy of identification and location of simple objects (i.e. 
stop signs). Firstly, mathematical morphology-based operations 
were applied to detect the region of interest. Then affine- 
invariant features were extracted and employed to identify the 
simple shapes of objects. Priese et al. (1994) and Sheng et al. 
(1994) both presented their own methods for traffic sign 
recognition. Color information was mainly used in their 
methods for object classification. It is recognized that automatic 
object recognition using image sequences has a long way to go. 
The reliable recognition of even simple objects is still 
challenging us. 
It is observed that, during the post-mission processing of the 
mobile mapping images, it is easy for a human operator to 
recognize an object from images, but it takes much time on 
browsing image sequences to find and locate the objects of 
interest. In the consideration of this fact, a database guided 
verification and updating of transportation objects with vertical 
line features was proposed and developed. This method is used 
to assist the user on updating the object database using image 
sequences. This is a novel idea on improving the efficiency of 
object measurement using knowledge aided feature extraction. 
b. Verification and updating of transportation objects with 
vertical line features 
The objective of this approach is to assist users on verifying the 
existence and checking the condition of existing transportation 
objects from an existing database, and further updating the 
geometric information (3-D coordinates) of the objects in the 
database (Tao, 1999). The systematic overview of the proposed 
approach is illustrated in Figure 4. It consists of three main 
modules: 
• Back projection of objects from a database onto a stereo 
image sequence 
• Verification of the existence of objects 
• Multinocular line reconstruction for accurate object 
positioning