Shih-Hong Chio
will use the first method. The requisite for using the first method is that the stereo correspondence of the corner in both
the left and the right image must be known. Therefore after the associated lines and corners have been selected or
positioned by the operator in the left image, the first question is whether the roof corners can be defined in the left
image. The second question is how to find the stereo correspondences in the right image.
In principle, if two adjacent edges of a corner have been confirmed in the left image, that corner can be determined by
the intersection of these two edges (unmarked corners in Fig.3). Moreover, if at least one of the two edges is a 3-D line
segment, then 3-D coordinates of the corner can be directly calculated without the need to find its stereo correspondence
in the right image. The calculation is based on simultaneous solving of Eq.1 and Eq.2, which will be explained later
again. If none of the two intersecting lines is 3-D lines, then that corner is only determined in the left image. Its stereo
correspondence needs to be determined in the right image.
x a mn-d]zx x,
y|=Aa2 52 e2] y |+| 1%, Eq.(1)
Z a3 5 cl-rt1z,
f : The Focal Length of Camera
X Y A : The 3-D Coordinates of the Projection Center
Xo Yu : The 3-D Coordinates of the Point in Object Space
al,a2,a3,b1,b2,b3,c1,c2,c3: The 9 direction cosines consists of the 3 angular elements
À : The Scale Factor
Y=aX+b Eq.(2)
X, Y: The Plane Coordinates of the Point in Object Space
a,b : Coefficient for Linear Equation in X-Y Plane
Only if one of the two adjacent edges to a corner is selected and the other one is missing (dotted lines in Fig.3), that
corner must be a lacking one (double circle in Fig.3). Of course, if both edges to a corner are missing, that corner is also
a lacking one. Due to the nature of the applied low-level feature extraction method [FOrstner, 1994] and the
simultaneous linking and matching of linear features in our system, lines and points are simultaneous extracted. The
presence of lines is independent of the presence of points. That is even though a line is missing there could be still
points existed at the lacking corner position. The points are stored in the Point Database. Therefore, even though no line
is available we still can go into the Point Database and look for points which might be roof corners.
For each lacking corner, the system will try to assign a point from the Point Database to that corner. If the operator is
not satisfied with the assignment, he can deny it. The system will assign another one. If there is no one available in the
Database, the current cursor position will be used directly as the corner. In Fig.3 all corners, which can not be
determined by the system itself, are marked as lacking corners
After all the corners have been determined in the left image, the system now examines the corners one by one to see if
the 3-D object coordinates of them could be solved from the information obtained so far. For example, all corners
related to the bottom edge in Fig.3 are solvable because the bottom edge is a 3-D line. Since the corner is requested to
locate on that line, the condition provides a height constraint. Therefore according to the second method mentioned
above, the 3-D object coordinates can be solved. Since the 3-D line segment might be only a portion of the entire roof
edge and the corner might not locate directly on the line, no direct height of the corner is used in solving the problem.
Instead, we will solve the 3-D coordinates of that corner by an indirect way. Two equations, Eq.1 and 2, are used for
solving the coordinates. Equation 1 states that all known pixels of the 3-D line segments together with the corner pixel
must satisfy the basic linearity equation. The difference is that for pixels of the 3-D line segment all 3 object coordinates
X, Y, Z are known, for the corner pixel its object coordinates are unknowns waiting to be solved. Equation 2 describes
the projection of this 3-D line on the X-Y plane of the object coordinate system. It says that the projection of the 3-D
line on the X-Y plane must be a straight line and the projection of the corner pixel must also on this straight line. For
each pixel including the corner pixel, one pair of Eq.1 and 2 can be listed. No matter how many pixels are included for
the computation there are always five unknowns, the two parameters a and b in Eq.2 and the X, Y, Z coordinates of the
corner pixel. Theoretically two pixels of the 3-D line and one corner pixel will give 6 equations which are enough for
solving the unknowns. But in general we have a lot more and the least squares adjustment must be applied. We can see
that any corner located on a 3-D line can be solved for it object coordinates in this way. But for all other cases the stereo
correspondence in the right image must be found first and the space forward intersection is used to calculate the 3-D
coordinates of the corner.
For the search of stereo correspondences in the right image the Point Database of the right image will be used. The
search for correspondence shall of course take place along the epipolar direction on the right image. But the question is
how to define the search range. Although there is no direct height information of the corner available, but from the
semantic information of the roof patch model and the semantic information of the initial 3-D line segment, we can
186 International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B3. Amsterdam 2000.