inbul 2004 
/ the color 
jects. The 
input data. 
) different 
| artificial 
ation and 
space. The 
on: (1) the 
image, (2) 
ace as (G- 
determine 
ts, shadow 
luce search 
to verify if 
1se a DSM 
he terrain, 
so-called 
aration of 
nd objects 
; not very 
n the DSM 
avias et al. 
vith image 
to confirm 
ompensate 
n. 
ans of the 
th of them 
roadmarks 
, while the 
1ey can be 
extraction 
centerlines 
rks and/or 
road sides 
es or city 
; using an 
ge line is 
)03a). The 
ct space by 
se that are 
o the road 
the buffer 
ks. Zebra 
sing color 
phological 
tripes. We 
Only the 
| ones arc 
vzed. The 
sines. The 
| crossings 
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol XXXV, Part B4. Istanbul 2004 
  
4. 3D ROAD RECONSTRUCTION 
With the extracted features and cues, our next step is to 
combine them to extract the road. Firstly, irrelevant edges are 
removed. 3D parallel overlapping edaes are then searched for, 
and are evaluated to find possible road sides. Due to occlusions 
or shadows, a road and its sides may be totally or partially 
invisible in images. The road segments with only one side 
visible in one or two images in the occluded/shadowed areas are 
inferred and reconstructed. In addition, gaps, where the road 
sides are totally invisible, are bridged. Finally, the road is 
reconstructed by finding an optimal path among the road 
segment candidates that maximizes a merit function. Highways, 
first class roads and most second class roads are also extracted 
using the detected roadmarks and the zebra crossings. With the 
extracted roads, the road junctions are reconstructed and 
modelled. The main procedures are described below. More 
details on all above procedures are given in Zhang (20032). 
4.1 Finding 3D parallel road sides 
The system checks the extracted edges to find 3D parallel edges. 
Only edges located in the buffer defined by the VEC25, having 
a similar orientation to the VEC25 segments and a certain slope 
are further processed. Edges above the ground are removed by 
checking with the nDSM. Two edges are considered as parallel 
if they have similar orientation in 3D space. The edges of a pair 
must overlap in the direction along the edges, and the distance 
between them must be within a certain range determined by the 
road class defined in the VEC25. In addition, the heights of the 
two edge segments should be similar. The found 3D parallel 
edges are projected onto the images and evaluated using 
multiple knowledge. The region between the projected edges 
must belong to the class road as determined by the image 
classification. If roadmarks are presented on this road, the 
extracted. roadmarks are used to confirm that the edge pair 
corresponds to correct road sides. The found 3D parallel edges 
have high probability of belonging to a road; they are called 
Possible Road Sides that are Parallel (PRSP). Thus, each PRSP 
is geometrically described by a pair of 3D straight edges and its 
corresponding 2D edges, and holds a set of attributes. 
42 Reconstruction of missing road sides 
Not all 3D road segments can be obtained from the procedures 
described above. The absence of 3D road sides can be caused 
by shadows, occlusions, or road sides do not actually exist, c.g. 
in the area where a parking lot is situated next to the road. 
Depending on the relations between the road segments and the 
neighbouring objects, sun angle, viewing direction, existence of 
moving cars on the road etc., there are various types of missing 
road sides in images. We have made an investigation and 
classified them into 11 types. Each type is then treated, and the 
missing road sides are inferred and validated by a specific 
procedure (Zhang, 2003a). Based on the type of the missing 
road sides, corresponding types of road segment candidates 
(RSCs) are obtained. They are then evaluated using the 
Knowledge obtained from the cues. Common reliability 
Measures for all RSCs are evaluated using the results from the 
Image classification and the nDSM. In addition, specific 
reliability measures for each type of RSC are also computed 
using the geometric relations between the RSC and its adjacent 
PRSPs. We refer to Zhang (2003a) for the detailed procedures 
of RSC evaluation. 
4.3 Gap bridging 
A gap between neighbouring PRSPs that belongs to a road rep- 
resents a road part where no road side is visible in the images. 
In our system, it is bridged either by directly linking the 
neighbouring PRSPs or by adapting the shape of the VEC25 
road corresponding to the gap area. That is, the vertices of the 
VEC25 road in the gap area are shifted to close the gap, based 
on the coordinate differences between the end points of the 
PRSPs and their corresponding points on the VEC25 road. 
Linking the adjacent PRSPs to close gaps is efficient on straight 
roads with short gap length, while using the shape of VEC25 
road might be more useful for long and curved occlusion areas. 
We determine the solution for gap bridging in an evaluation 
process using various knowledge, e.g. the shape of the solution 
for the gap should approximately comply with that of the 
VEC25 road; it should be either a road region or a shadow or 
shadow mixed with road region; or roadmarks are extracted 
within the hypothesised road area. Based on the evaluation, we 
compute a measure for the gap hypothesis, s,,,. The range of 
values for s,,, is [0,1], with decreasing value for long and 
Inconsistent gaps. 
4.4 Road segment linking for 3D road reconstruction 
With the extracted PRSPs, the road is reconstructed by linking 
the PRSPs belonging to a road. The goal of linking road 
segments is twofold. First, this implies that PRSPs belonging to 
a road should be selected and connected with the gaps bridged 
by the linking algorithm. Secondly, this also implies that PRSPs 
not belonging to a road should be rejected. Therefore, the 
algorithm must be very selective in which PRSP it adds to a 
road. The linking function in our system is defined as 
> les, + eap Sop +1; "5; 
(1) 
where, i and j are adjacent PRSPs, /; and I; are their lengths, s; 
and s; are their reliability measures. l,, is the gap length 
between 1 and j, and Sgap 1S the gap evaluation measure. The 
function takes high values for long curves with a shape similar 
to the VEC25 road. The linking problem can then be solved by 
finding a subset among all PRSPs that maximizes the linking 
function. This can be achieved using dynamic programming 
(Grün and Li, 1997). 
Higher class roads are also extracted using the detected 
roadmarks and zebra crossings. The roadmarks are linked using 
a similar method as described in the previous paragraph. This 
procedure increases the effectiveness and reliability of our 
system. In complex areas, such as in city centers, the road sides 
are generally occluded very much, while sometimes they are not 
defined. However, the road centerlines are successfully 
extracted by the system using roadmarks. In rural and suburban 
areas, the extracted road using roadmarks is used by the system 
to verify the extraction results using 3D parallel edges. 
4.5 Road junction generation and modelling 
Road junctions are important features of the road network. 
However, it is even more difficult to model and extract road 
junctions from images than road segments. This might be one of 
the reasons that this issue has been rarely touched in past 
rescarch. In our system, we reconstruct junctions through 
intersecting the extracted roads guided by the topology of the 
VEC25 data, and further model the junctions with road class 
information and the shape of the VEC25. During the process, 
each road is assigned a weight corresponding to the evaluation 
1055