next 
to be 
rward 
) fea- 
same 
s the 
mage 
which 
sured 
| been 
e sit- 
(right 
ig.8). 
is ap- 
inter- 
y the 
resent 
tivity 
anges 
called 
:d for 
juares 
dicted 
h, the 
is by- 
in the 
ls see 
there. 
dvan- 
d for 
n the 
ye) to 
| done 
edge, 
poral 
sible, 
as of 
in an 
an be 
ternal 
actual 
^ature 
terest 
sition 
hicle. 
ion in 
wheel 
(0). These aspects yield the dynamical model as described 
in [Hock 90a]. 
After linearization and with the help of standard 
  
  
  
Fig.9: Dynamical model 
methods of modern control theory the discrete state tran- 
sition form is derived (see fig.9). 
Geometric model 
The geometric properties of the scene are exploited in 
combination with the laws of perspective projection in 
order to describe the position of relevant features in the 
image plane as a function of relative spatial state. The 
landmarks are modeled as 3D objects with known coordi- 
nates of their centroid and the spatial feature distribution 
relative to this. The perspective projection equations give 
  
Fig.10: Geometrical model 
the horizontal coordinate yp; and the vertical coordinate 
zpi 0f the landmark L; as measured in the image plane (see 
fig.10). 
Recursive state estimation 
The dynamical models link time to spatial motion, in 
general. 3D shape models exhibit the spatial distribution 
of visual features which allow to recognize and track 
objects. In order to exploit both dynamical and shape 
models at the same time, the prediction error feedback 
scheme for recursive state estimation developed by Kal- 
man and successors in the 60-ies has been extended to 
image sequence processing by our group [Wuensche 86; 
88]. There are many publications on this approach so that 
only a short summary will be given here ( see e.g. the 
survey articles [Dickmanns, Graefe 88; Dickmanns, Mys- 
liwetz 92]). The Kalman filter approach introduces knowl- 
edge about the dynamical behavior of a process, about the 
measurement relations and about noise statistics of both 
process and measurements in order to obtain best esti- 
mates of the process states in a least squares error sense 
recursively as new measurement data arrive. Iteven allows 
to substitute this knowledge for missing measurements of 
state components; these are reconstructed in a way to best 
fit the overall model. In the 4D-approach to dynamic 
vision, the Extended Kalman Filter (EKF) for nonlinear 
systems (see [Maybeck 79]) has been further extended to 
perspective mapping as the measurement process; the 
reconstruction capability is thereby exploited for bypass- 
ing the strongly nonlinear perspective inversion, utilizing 
all continuity conditions for spatio-temporally represented 
objects in 3D space (shape, carrying well visible features) 
and time (motion constraints, given by the dynamical 
model, the differential equations of motion). 
State estimation, as used here, plays a dual role in the 
visual interpretation process: 
First, it yields a direct transformation from feature 
locations in image sequences into physical quantities in 
space (such as x, y, V and their time derivatives), which 
are related to control actuations. 
Second, when using this approach also the control in- 
puts (u) of the vehicle can explicitly be taken into account. 
Via known dynamics (state transition matrix ®) the sys- 
tem's state x" at the next sampling time can be predicted, 
thus also the expected appearance of landmarks can be 
computed as a vector y . This information is used directly 
to guide the feature extraction process where to look for 
edges or lines of tracked landmarks. 
Only those features matching best the predicted location 
will be selected and used to actually drive the interpreta- 
tion process. The selection step is augmented by the infor- 
mation contained in the estimation error covariance matrix 
P. Mapping the predicted uncertainty of the state estimates 
into measurement space yields the innovation variance, 
which defines the allowed neighborhood of the predicted 
values y" in which the new incoming measurements 
should lie. Based on this information and by processing 
only single measurements sequentially, outliers can be 
rejected. This selection capability reduces measurement 
noise and is crucial for the robustness of the approach 
under real-world conditions. 
It should be noted, that the measurement equations have 
to be evaluated only in the forward direction, from state 
space into the image plane. The non-unique inverse per-