next
to be
rward
) fea-
same
s the
mage
which
sured
| been
e sit-
(right
ig.8).
is ap-
inter-
y the
resent
tivity
anges
called
:d for
juares
dicted
h, the
is by-
in the
ls see
there.
dvan-
d for
n the
ye) to
| done
edge,
poral
sible,
as of
in an
an be
ternal
actual
^ature
terest
sition
hicle.
ion in
wheel
(0). These aspects yield the dynamical model as described
in [Hock 90a].
After linearization and with the help of standard
Fig.9: Dynamical model
methods of modern control theory the discrete state tran-
sition form is derived (see fig.9).
Geometric model
The geometric properties of the scene are exploited in
combination with the laws of perspective projection in
order to describe the position of relevant features in the
image plane as a function of relative spatial state. The
landmarks are modeled as 3D objects with known coordi-
nates of their centroid and the spatial feature distribution
relative to this. The perspective projection equations give
Fig.10: Geometrical model
the horizontal coordinate yp; and the vertical coordinate
zpi 0f the landmark L; as measured in the image plane (see
fig.10).
Recursive state estimation
The dynamical models link time to spatial motion, in
general. 3D shape models exhibit the spatial distribution
of visual features which allow to recognize and track
objects. In order to exploit both dynamical and shape
models at the same time, the prediction error feedback
scheme for recursive state estimation developed by Kal-
man and successors in the 60-ies has been extended to
image sequence processing by our group [Wuensche 86;
88]. There are many publications on this approach so that
only a short summary will be given here ( see e.g. the
survey articles [Dickmanns, Graefe 88; Dickmanns, Mys-
liwetz 92]). The Kalman filter approach introduces knowl-
edge about the dynamical behavior of a process, about the
measurement relations and about noise statistics of both
process and measurements in order to obtain best esti-
mates of the process states in a least squares error sense
recursively as new measurement data arrive. Iteven allows
to substitute this knowledge for missing measurements of
state components; these are reconstructed in a way to best
fit the overall model. In the 4D-approach to dynamic
vision, the Extended Kalman Filter (EKF) for nonlinear
systems (see [Maybeck 79]) has been further extended to
perspective mapping as the measurement process; the
reconstruction capability is thereby exploited for bypass-
ing the strongly nonlinear perspective inversion, utilizing
all continuity conditions for spatio-temporally represented
objects in 3D space (shape, carrying well visible features)
and time (motion constraints, given by the dynamical
model, the differential equations of motion).
State estimation, as used here, plays a dual role in the
visual interpretation process:
First, it yields a direct transformation from feature
locations in image sequences into physical quantities in
space (such as x, y, V and their time derivatives), which
are related to control actuations.
Second, when using this approach also the control in-
puts (u) of the vehicle can explicitly be taken into account.
Via known dynamics (state transition matrix ®) the sys-
tem's state x" at the next sampling time can be predicted,
thus also the expected appearance of landmarks can be
computed as a vector y . This information is used directly
to guide the feature extraction process where to look for
edges or lines of tracked landmarks.
Only those features matching best the predicted location
will be selected and used to actually drive the interpreta-
tion process. The selection step is augmented by the infor-
mation contained in the estimation error covariance matrix
P. Mapping the predicted uncertainty of the state estimates
into measurement space yields the innovation variance,
which defines the allowed neighborhood of the predicted
values y" in which the new incoming measurements
should lie. Based on this information and by processing
only single measurements sequentially, outliers can be
rejected. This selection capability reduces measurement
noise and is crucial for the robustness of the approach
under real-world conditions.
It should be noted, that the measurement equations have
to be evaluated only in the forward direction, from state
space into the image plane. The non-unique inverse per-