1d on the 
e second 
i.e. object 
efined by 
section of 
thermore, 
tersection 
own from 
he object 
ived from 
vo object 
forward 
lel planes 
planes is 
ance to be 
ray of an 
em. After 
(6) 
ns (2) and 
1e epipole 
e, forward 
projection 
d onto the 
e / and 
(7) 
direction, 
For the 
t image is 
yuld yield 
X the two 
(8) 
ed in (van 
ed form of 
ences the 
. Each test 
pole. The 
ndences of 
and at the 
verify the 
ntersection 
al to the 
ps: 
her one of 
n existing 
cluster is 
esent in an 
becomes a 
  
candidate for that cluster. 
e The candidate correspondence becomes a member of the 
cluster if all tests of constraints (3) are accepted. These 
tests involve three correspondences: the candidate and 
two correspondences already in the cluster. 
This procedure is repeated as long as new clusters are 
created. The result is that overlap between clusters can be 
considerable. Many of these clusters contain a set of correct 
correspondences with minor differences. The clusters that 
contain more than a minimum number of correspondences 
are analysed in more detail by applying an integrated 
adjustment of all constraints. 
3.3.5 Overall adjustment and testing 
The clustering procedure results in groups of 
correspondences that support the same relative orientation, 
while the related object points are expected to recede in the 
same object plane. An overall adjustment is set up for all 
clusters with more than a minimum number of 
correspondences. The functional model contains 7-1 
condition equations for coplanarity (equation (8), n is the 
number of correspondences), and n-2 condition equations for 
intersection of epipolar planes (equation (3)). A set of 
independent equations results. The adjusted epipole is 
computed from adjusted observations. Two different types of 
statistical tests are applied. First an overall test or Fisher test 
is applied. The second test examines the alternative 
hypothesis of an error in a single correspondence. If such a 
test is rejected, the correspondence is removed from the 
cluster and the model is built again. This iterative testing 
procedure stops if all correspondence tests are accepted. The 
cluster with the largest number of correspondences of which 
no correspondence tests are rejected is selected. This cluster 
is expected to contain corresponding image points of which 
the related object points are in (or near to) a plane. More 
details on the statistical testing can be found in (van den 
Heuvel, 1998). 
3.3.6 Towards automatic reconstruction 
Apart from the primary cluster detected as described above, 
other clusters can also contain correct correspondences of 
which the related object points are in a different plane. In 
order to detect such clusters, all clusters that meet the 
following two requirements are incorporated in the overall 
adjustment: 
1. The cluster does not have a correspondence (nor an 
image point) in common with the primary cluster. 
2. The correspondences of the cluster confirm the epipole 
of the primary cluster. 
For all correspondences — those of the primary as well as 
those of the clusters selected by the criteria above — the 
epipolar plane intersection constraints are set up. Of course, 
the coplanarity constraints are only applied to 
correspondences of the same cluster. 
With the integrated adjustment of more than only the primary 
cluster, there is not only additional evidence gathered for the 
epipole, but at the same time different object planes are being 
detected. Preliminary faces can be created by a bounding box 
around the object points of a cluster. Object planes are then 
to be intersected to find the edges of the building. The 
detection of object planes is a by-product of the proposed 
procedure for automatic relative orientation, but also a first 
step towards automatic reconstruction. The experiments 
described in the next section aim at the detection of the 
primary object plane only. 
—231— 
4. EXPERIMENTAL RESULTS 
In this section an experiment is discussed in which the 
procedure for automatic relative orientation is applied to 
three images of a historic building. The images are taken 
from ground level with a handheld calibrated digital camera 
(1536x1024 pixels). They were taken from the south-south- 
west (SSW), south-east (SE), and east (E) approximately 
(Figure 5). The number of extracted straight lines can be 
found in Table 1. 
  
Figure 5: The three images (labelled SSW, SE, and E) 
The a priori precision of the endpoints of the lines was set to 
1 pixel standard deviation in the vanishing point and the 
epipole detection. The "vanishing lines" are the lines that 
were uniquely grouped to one of the three vanishing points. 
These lines are displayed in Figure 6. Especially near the 
horizon line of image SE a considerable number of lines is 
lost because the vanishing point detection cannot distinguish 
between the left and right facade. As a result, for this image 
most of the intersection points are created in the upper part of 
the facades. Some statistics of the epipole detection are 
listed in Table 2. 
  
SSW | SE E 
# lines 339 457 | 202 
# vanishing lines 286 276 143 
# intersection points 164 78 30 
  
  
  
  
  
  
  
  
  
Table 1: Numbers of extracted lines and points 
  
  
  
  
  
  
  
  
  
  
  
SSW - SE SE-E 
# correspondence hypotheses 1109 144 
# coplanarity tests 182899 3308 
# accepted tests 15825 1102 
# clusters 5638 424 
max. # points in a cluster 16 9 
# clusters accepted 97 85 
max. # points in a cluster 15 7 
Fisher-test / critical value 4.4 1.0 
Deviation manually measured 1.7 deg 1.6 de 
  
Table 2: Statistics of the epipole detection 
The detected correspondences are displayed in Figure 7. Note 
that for the first image pair only those corresponding points 
are detected that are on the central part of the facade because 
this part is in a different plane from the rest of the facade. In 
fact, all possible correspondences are detected. However, 
looking at the location of the image points in detail, two 
corresponding points are often not at exactly the same 
location on the building. The reason is that many edges 
border occlusions. When there are several points created 
close together — which often is the case in the corners of the 
windows — the statistical testing cannot distinguish between 
different possible correspondences. Indeed, many of the 
accepted clusters are very similar in the correspondences 
they contain and their Fisher-test. Furthermore, as a result of 
a large number of "imperfect" correspondences (the image 
points are not projections of exactly the same point on the 
ADU oc LEEREN LE EEE