Shih-Hong Chio 
  
will use the first method. The requisite for using the first method is that the stereo correspondence of the corner in both 
the left and the right image must be known. Therefore after the associated lines and corners have been selected or 
positioned by the operator in the left image, the first question is whether the roof corners can be defined in the left 
image. The second question is how to find the stereo correspondences in the right image. 
In principle, if two adjacent edges of a corner have been confirmed in the left image, that corner can be determined by 
the intersection of these two edges (unmarked corners in Fig.3). Moreover, if at least one of the two edges is a 3-D line 
segment, then 3-D coordinates of the corner can be directly calculated without the need to find its stereo correspondence 
in the right image. The calculation is based on simultaneous solving of Eq.1 and Eq.2, which will be explained later 
again. If none of the two intersecting lines is 3-D lines, then that corner is only determined in the left image. Its stereo 
correspondence needs to be determined in the right image. 
x a mn-d]zx x, 
y|=Aa2 52 e2] y |+| 1%, Eq.(1) 
Z a3 5 cl-rt1z, 
f : The Focal Length of Camera 
X Y A : The 3-D Coordinates of the Projection Center 
Xo Yu : The 3-D Coordinates of the Point in Object Space 
al,a2,a3,b1,b2,b3,c1,c2,c3: The 9 direction cosines consists of the 3 angular elements 
À : The Scale Factor 
Y=aX+b Eq.(2) 
X, Y: The Plane Coordinates of the Point in Object Space 
a,b : Coefficient for Linear Equation in X-Y Plane 
Only if one of the two adjacent edges to a corner is selected and the other one is missing (dotted lines in Fig.3), that 
corner must be a lacking one (double circle in Fig.3). Of course, if both edges to a corner are missing, that corner is also 
a lacking one. Due to the nature of the applied low-level feature extraction method [FOrstner, 1994] and the 
simultaneous linking and matching of linear features in our system, lines and points are simultaneous extracted. The 
presence of lines is independent of the presence of points. That is even though a line is missing there could be still 
points existed at the lacking corner position. The points are stored in the Point Database. Therefore, even though no line 
is available we still can go into the Point Database and look for points which might be roof corners. 
For each lacking corner, the system will try to assign a point from the Point Database to that corner. If the operator is 
not satisfied with the assignment, he can deny it. The system will assign another one. If there is no one available in the 
Database, the current cursor position will be used directly as the corner. In Fig.3 all corners, which can not be 
determined by the system itself, are marked as lacking corners 
After all the corners have been determined in the left image, the system now examines the corners one by one to see if 
the 3-D object coordinates of them could be solved from the information obtained so far. For example, all corners 
related to the bottom edge in Fig.3 are solvable because the bottom edge is a 3-D line. Since the corner is requested to 
locate on that line, the condition provides a height constraint. Therefore according to the second method mentioned 
above, the 3-D object coordinates can be solved. Since the 3-D line segment might be only a portion of the entire roof 
edge and the corner might not locate directly on the line, no direct height of the corner is used in solving the problem. 
Instead, we will solve the 3-D coordinates of that corner by an indirect way. Two equations, Eq.1 and 2, are used for 
solving the coordinates. Equation 1 states that all known pixels of the 3-D line segments together with the corner pixel 
must satisfy the basic linearity equation. The difference is that for pixels of the 3-D line segment all 3 object coordinates 
X, Y, Z are known, for the corner pixel its object coordinates are unknowns waiting to be solved. Equation 2 describes 
the projection of this 3-D line on the X-Y plane of the object coordinate system. It says that the projection of the 3-D 
line on the X-Y plane must be a straight line and the projection of the corner pixel must also on this straight line. For 
each pixel including the corner pixel, one pair of Eq.1 and 2 can be listed. No matter how many pixels are included for 
the computation there are always five unknowns, the two parameters a and b in Eq.2 and the X, Y, Z coordinates of the 
corner pixel. Theoretically two pixels of the 3-D line and one corner pixel will give 6 equations which are enough for 
solving the unknowns. But in general we have a lot more and the least squares adjustment must be applied. We can see 
that any corner located on a 3-D line can be solved for it object coordinates in this way. But for all other cases the stereo 
correspondence in the right image must be found first and the space forward intersection is used to calculate the 3-D 
coordinates of the corner. 
For the search of stereo correspondences in the right image the Point Database of the right image will be used. The 
search for correspondence shall of course take place along the epipolar direction on the right image. But the question is 
how to define the search range. Although there is no direct height information of the corner available, but from the 
semantic information of the roof patch model and the semantic information of the initial 3-D line segment, we can 
  
186 International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000.