Christoph Kaeser 
  
3D edge generation is a crucial component of our procedure. We are interested in 3D straight lines because they are 
prominent in most man-made environments, and usually correspond to objects of interest in images, such as buildings 
and road segments. They can be detected in image data, and they provide a great deal of information about the structure 
of the scene. Additionally, since edge features have more support than point features, they can be localised more 
accurately. The 3D information of straight lines is determined from the correspondences of line segments between two 
images. Edges are derived by the Canny operator and subsequent straight line fit (Henricsson, 1996). The developed 
method exploits rich line attributes and line geometrical structure information. The rich line attributes include the 
geometrical description of the line (position, length, orientation) and the photometric information in the regions left and 
right of the line (flanking regions) using the Lab colour space. With known orientation parameters, the epipolar 
constraint can be employed to reduce the search space. The comparison with each candidate edge is then made only in 
the common overlap length, i.e. ignoring length differences and shifts between edge segments. For each pair of lines, 
which satisfy the epipolar constraints above, their rich attributes are used to compute a similarity score. The similarity 
score is a weighted combination of various criteria. All the scores are from 0 to 1, and the total similarity score is the 
average of all scores. The similarity score computation starts from the longer lines, while the very short ones (« 5 
pixels) are ignored. After performing similarity measurement computation, we construct a matching pool and attach a 
similarity score to each line pair. However, one still has problems to determine the best matches. In addition, matching 
using a very local comparison of line attributes does not necessarily give results consistent in the neighbourhood. 
Locally consistent matching is achieved through structural matching of neighbouring edges with probability relaxation, 
whereby the similarity scores serve as prior information. Structural matching is conducted bidirectionally from left to 
right and right to left. The next step is a combined 2D and 3D grouping of straight segments. Thereby, information in 
one space helps bridging gaps and combining segments in the other space. Thus, small gaps are bridged, edges broken 
in multiple straight segments are combined, matched segments of different length are extended. The final 3D position is 
computed from the original edge pixels and not the fitted straight lines. A 3D straight line (or polyline) is then fitted to 
the 3-D points. 
The proposed method for straight line matching is implemented and experiments have been performed on a number of 
areas extracted from aerial images. The test areas cover different terrain and landcover, including rural areas, suburban, 
urban, and hills. Figures 7-9 show an example. The line extraction process resulted in 985 and 971 straight lines in the 
left and right images respectively (see Fig. 8). Only 789 and 809 lines in the left and right images are longer than 5 
pixels. 460 matches are found, of which only 5 are wrong matches. The matching result is shown in Fig. 9. 
Figure 6. The test image pair. Figure 7. Extracted straight lines. Figure 8. The matching results. 
Besides the work on straight line matching and 3D line generation, we completed a multispectral image classification 
method to find road regions. Guided by the initial knowledge base, we excluded the lines outside the road buffer area 
(this area is defined using the road centerlines of VECTOR25 and their estimated maximum error). By combining the 
2D lines with the classification result, a relation with the road region (in, outside, at the border) is attached to each line. 
Lines with a slope difference to the slope from the known local DHM25 larger than a certain value are excluded. 
6 OUTLOOK 
We presented a new scheme for road and building reconstruction from aerial images. The proposed idea uses as much 
information as possible to increase success rate and reliability of the results. The project still needs significant work to 
be successfully completed. The most difficult part, the combination of all available information, disambiguation of 
conflicts and derivation of reliable quality criteria, lies ahead. Use of more than two images will be made with our test 
  
468 International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000.