7A-4-4 
(2.b) Take this edge as a starting point, and construct a vertical 
“bridge” downwards, with a width of 3 pixels, and a length of 
30 pixels. 
(2.c) Same as step (1 .c). 
(2.d) If the detected line segment is longer than 1/3 of the length of 
its associated primary line segment, the parameter “parallel” in 
the line file will be set as ‘True”, otherwise “False”. The file 
structure of primary lines will be described later. 
This grouping algorithm represents a good trade-off. The first phase 
is used to extract reliable and distinct line segments from the images, 
thus the bridging is relatively strict. While the second phase is 
considered as an evidence collection, so that a large length of 
bridging gap is applied to collect evidences from noisy data. The 
results of the line grouping are illustrated in Figure 3e. 
3.2.2 Representation of Line Segments 
The detected line segments have to be represented symbolically in 
order to facilitate subsequent processing. A line description file is 
generated after line grouping. The structure of this file is as shown in 
Table 1. All the parameters of line features are computed and 
recorded. The starting and ending point positions, and the length of 
the line are two basic feature parameters. The number of edges refers 
to the number of compatible edges in the line. Line direction is 
defined as the average direction of all the compatible edges in the 
line. “Stereo True” means that the corresponding line segment in the 
right image is found. The establishment of the stereo 
correspondences will be described in the next section. “Parallel 
True/False” indicates the state whether or not a secondary line 
segment of this line is found. 
3.3 Feature Correspondence of Line Segments 
The algorithms described above are applied onto all images 
containing the object to be examined. At this step, feature 
correspondence of line segments will be performed to further 
verify the existence of the vertical objects. If a line segment 
extracted above has a correspondence in another image, this line 
segment is very likely to be a part of a vertical object. To this 
end, the most distinctive line is chosen from the line file 
generated above. According to the parametric representation, a 
sorting algorithm is used to select this line. The sorting order is 
“Parallel True” -> “Line length” -> “Number of edges”. High 
priority is given to the line segment whose parallel line segment 
has been found. Then the line length is considered, the longer 
the better. Lastly, the number of compatible edges is taken into 
account. 
The image containing the most distinct line is treated as a 
master image, while its corresponding stereo image is treated as 
a slave image. In the correspondence method, three constraints 
are used. The first constraint is the direction compatibility. The 
corresponding line in the slave image exists only if its line 
direction is the same as the line direction in the master image 
within a tolerance ¿10°. 
The second constraint comes from the scene knowledge. First of 
all, the disparity range constraint is applied (Tao, 1996 and Tao 
et al., 1997). Besides the disparity range constraint, a road 
corridor condition is also used to eliminate the undesired line 
features. It is impossible that vertical objects of interest are 
located right in the middle of a road. Therefore, the distance 
from an object to the vehicle trajectory should be more than 1 m 
(the camera baseline is 2 m). This constraint is useful to screen 
out the detected features which are associated with the objects 
on the road. 
The third constraint is the similarity of line segments. We use 
the weighted cross-correlation to perform similarity matching 
(Tao, 1997). As illustrated in Figure 6 (a part of Figure 3e), the 
matching procedure is described as follows: 
(a) pick a detected line segment from the master image, and 
then generate a set of sampling points along the line 
segment with an interval of 3 pixels; 
(b) each sampling point is treated as the center of the master 
matching window (size of 15H x 1IV); 
(c) generate the corresponding epipolar line in the slave 
image and determine the intersection of the epipolar line 
and the vertical line segments detected in the slave image. 
These vertical line segments must satisfy the previous two 
conditions, namely, line direction and disparity range 
constraints; 
(d) take the intersection point as the center of the search 
window (size of 21H x 1IV); 
(e) if the computed correlation coefficient surpasses the 
threshold (0.7), record this match point; 
(f) once all the sampling points are used, the corresponding 
line in the slave image exists, if the number of match 
points on that line is more than half of the total number of 
sampling points used; 
(g) finally, update the parameter of “Stereo True/False” in the 
associated line file. 
Figure 6. Stereo correspondence ofline segments 
If the stereo correspondence of the line segment can be 
established, it has a great possibility to say that the object, for 
instance, stop sign, does exist. In the next, the position of the 
object will be determined using a multinocular line 
reconstruction technique. 
4. MULTINOCULAR LINE RECONSTRUCTION FOR 
OBJECT POSITIONING 
Multinocular epipolar geometry is applied here to improve the 
reliability of line reconstruction. After stereo correspondence, 
the 3-D coordinates of the sampling points along the line (in the 
master image) can be calculated. These 3-D points can be back 
projected in the third image, say the forward left image. A 
multiple image matching method along with the consistency 
filtering technique based on the multinocular epipolar constraint 
is applied. As shown in Figure 7 (the match points have been 
marked by ‘-’), the corresponding point in the third image 
(forward-left image) can be determined if it locates at the 
intersection of multiple epipolar lines. For more detailed