International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B5. Istanbul 2004 
and Spann, 1999). Thus, the threshold value could be set 
automatically by following equation. 
S9S82u70, (5) 
{ max 
where, 
= 2 ac \ 
| {8S a /@S 2 
Gm "Gut EE "Op 
OD) cP 
Spax 1S maximum value of S, and c5 and c» are accuracy for D 
and P. Therefore, S, which is threshold value of S can be 
calculated automatically, and the line matching can be 
performed. 
3. LINE MATCHING BY TRIFOCAL TENSOR 
Some unmatched lines will remains in the above optical flow 
estimation procedures. In order to revive unmatched lines, 
trifocal tensor was adopted in this paper. Details of the line 
matching by trifocal tensor are as follows. 
3.1 Trifocal Tensor 
Trifocal tensor is geometric relation between 3 images which 
were taken from different camera positions for the same object. 
The trifocal tensor is expressed by 3 square matrixes (3x3), 
these 3 matrixes are T,, T; and T4, components of these 
matrixes are 1,;, fo And £5j, and image coordinates of matched 
common points to these 3 images are (xi, yi, zi), (x2. y». z2) and 
(Xs, ys, zi) Thus, following equations are obtained by the 
geometric relation. 
—ZZ,83 t 2, V:85n + Va23E 7 — VaYs833 = 0 
2,582 7 Z2X3823 7 V27358 7 + VaX3833 = 0 (6) 
2224812 — ZaV3B13 7 X27387 TV En = 0 
— 2421841 + Z2X3815 + X2238 a 7 XaX3833 = 0 
where, 
Sj E Xf x yis + zt 
3j 
These 4 equations are generated by one common conjugated 
point to 3 images. The trifocal tensor has 27(-3x3x3) unknown 
parameters which are able to calculate by more than the same 
number of equations. Therefore, more than 7 common 
conjugated points are needed to 3 images, and unmatched 
points in the third image are calculated by these above 
equations. 
3.2 Least-Median Squares (LMedS) 
In order to acquire unknown parameters by observation 
equations, calculation by least squares method is used. However, 
observation equations for the trifocal tensor are only acquired 
for common conjugated points to 3 images, and calculation of 
the trifocal tensor depend on the accuracy of the matched line. 
Therefore, calculation of the trifocal tensor by Least-Median 
Squares (LMedS) (Rousseeuw and Leroy, 1986) was 
investigated in this paper. The LMedS is one of the numerical 
calculation method by least squares using only accurate 
observation equations. Detail procedures of the LMedS are as 
follows: 
(1) The trifocal tensor is calculated by these observation 
equations using least squares method, and observation 
errors for each equation which are generated by least 
squares are acquired. 
(2) A median value for these observation errors is extracted, 
and threshold value for the observation error 6 is 
calculated by following equation. 
  
euch. 3 [mede (7) 
| n=F 
where, 
C : coeficient value (= 1.4826) 
n : number of observation equations 
F : number of unknown parameters 
6; : observation error 
(3) These observation equations which have the observation 
error more than the threshold value are removed, and the 
trifocal tensor is calculated by least squares using out of 
those equations. 
(4) Above procedures are iterated until observation errors for 
all equations become less than the threshold value, and 
final result is adopted as accurate trifocal tensor. 
In addition, lines which were corresponded for the removed 
equations were also removed during the line matching 
procedure. Consequently, the trifocal tensor could be calculated 
accurately, and useless lines could be removed efficiently. 
4. EPIPOLAR MATCHING 
The line matching was performed efficiently by the above 
procedures. However, these procedures can not apply for all 
necessary lines due to fragment or multiple. Therefore, the 
unmatched lines were corrected using epipolar matching. 
The epipolar matching was performed using epipolar lines for 
the first and last image. In order to estimate epipolar lines, 
relative orientation was performed by coplanarity condition 
using the first and last image. The both ends for the each 
matched lines were used as pass points, and the orientation 
parameters (9j, Kj, @2, 9», K;) Were determined. After the 
orientation, geometric correction of the first and last image was 
performed using the orientation parameters. Consequently, 
epipolar lines were estimated. 
Furthermore, in order to perform stereo matching by using these 
epipolar lines efficiently, least squares matching (LSM) method 
was adopted in this paper (Gruen, 1985). 
As a result, line matching for 81 lines could be performed 
correctly. These matched lines constitute surface of Koma 
house, and out of those useless lines such as background or 
timber were removed automatically. Figure 6 shows the result 
of the line matching. 
5. 3D MODELLING 
The line information for 3D modelling can be acquired 
efficiently by the method in the previous chapter. However, 
each surface on the Koma house is needed to be recognized for 
3D modelling. Therefore, surface recognition was performed by 
morphological opening procedure, and the extracted surfaces 
were conjugated with the matched lines in this paper (Kunii and 
Chikatsu, 2003). Figure 7 shows the result of the opening 
procedure for the first image. 
   
Interne 
Furthe 
house, 
were | 
2002). 
calcul: 
genera 
frame 
perfon 
and Fi