So, noting the complication that non-orthogonal viewing introduces to the analysis, we wish 
to point out that the approach outlined earlier, although expediting the explanation, is not 
the way that we currently handle the analysis. We do not find linear feature paths in 
the EPI images. A desire to handle varied camera geometries has lead us to a reconsideration 
of the way we process the data. Recent thesis research of Marimont ([Marimont 19862], also 
see [Marimont 1986b]) has lead to a generalization of our analysis that unifies the treatment 
of camera geometries for all camera angles with respect to the linear trajectory. The basis of 
the generalization comes from the observeration that no matter where a camera roams about a 
scene, for any particular feature, the lines of sight from the camera principal point through that 
feature in space (determined by the line from the principal point through the image plane point 
where the projected feature is observed) all intersect at the feature point (ignoring measurement 
error). Calling upon duality (see | 
[Coxeter 1964], and Figure 14), note \ 
that the duals of the lines of sight are [ns \ 
points which lie along a line in their \ The DUAL GE a, ine \ 
dual space, and the dual of this line is A SA the original scene point 
    
the feature point in the original carte- 
sian space of the scene. Fitting lines of 
sight to the feature is equivalent to fit- 
ting a line to the points that are dual 
to the lines of sight. 'This means that 
feature tracking in the space of lines of 
sight is linear. The error metric used 
in the minimization is an important 
consideration, and because of this, we 
do the analysis in the space of lines of 
sight, rather than in the dual. In sum- 
mary, the principal advantage of using 
the line of sight formulation is that, 
The DUALS of lines of sight 
here, ALL paths of stationary features i tim 
are linear, regardless of viewing direc- | i 
tion, none are hyperbolic, so detecting SCENE SPACE DUAL SPACE 
and grouping observations of particu- | 
lar features is very much simplified. Fig. 14. Duality 
3: Experimental Results 
  
We have developed a system that carries out the above analysis, computing 3-space locations of 
scene features by analyzing spatio-temporal volumes produced during linear camera motions. 
The processing currently consists of the following steps: 
1. 3D convolution of the spatio-temporal data 
. Slicing the convolved data into EPIs 
Detecting edges in the EPIs and converting them to lines of sight 
Segmenting these lines of sight into linear pieces 
Merging collinear pieces 
Computing x-y-z coordinates 
Building à map of free space 
Linking x-y-z points between EPIs 
Qo 21 QU co bo 
In this section we illustrate the performance of the system by applying it to two data sets. We 
begin with the data shown in Figure 2. 
The first step processes the three-dimensional data to determine the spatio-temporal contours 
subsequently to be used as features (and, incidently, to reduce the effects of noise and camera 
jitter). This is done by forming a 3-dimensional difference of Gaussians ([Buxton 1983] and 
[Buxton 1985] explore other uses of spatio-temporal convolution). 
The second step forms EPIs from the convolved spatio-temporal data. For a lateral motion this 
is straightforward because the EPIs are horizontal slices of the data. Figure 9 shows the EPI 
selected to illustrate steps three through seven. This slice contains a plant on the left, a shirt 
draped over a chair, part of the top of a table, and in the right foreground, a ladder. 
124