MOVING HUMANS RECOGNITION USING SPATIO-TEMPORAL MODELS” 
Winfried Kinzel and Ernst D. Dickmanns 
Institut für Systemdynamik und Flugmechanik 
Universität der Bundeswehr München (UBM) 
Werner-Heisenberg-Weg 39 
D-8014 Neubiberg, Germany 
phone +49-89/6004-3583 
fax +49-89/6004-4092 
ISPRS Commission V 
ABSTRACT 
A computational framework for dynamic scene 
analysis with respect to 3D-real-time tracking of 
human motion in typical road environment is 
presented. In contrast to the practice of many pattern 
recognition techniques which refrain from considering 
the underlying signal process, this approach utilizes a 
modified observer concept from control theory to keep 
track of changing image features and to aggregate 
them in a deductive rather than an inductive manner. 
A procedure of recursive estimation of limb states is 
derived for humans modelled as mechanical multibody 
system; it is supported and updated by feature based 
image sequence processing. For system development 
with versatile input signals and for an assessment of 
estimator performance an extensive animation tool has 
been designed. The proposed approach requires 
moderate computational power so that the complex 
recognition task may be accomplished in real-time in 
the near future. It promises less redundance and may 
serve as a simulation of perception in general. 
Key Words: Human Motion, Computer Graphics, 
Image Analysis, Object Recognition, Feature 
Tracking, Recursive State Estimation. 
1. INTRODUCTION 
1.1. Motivation 
The project objective towards ^ automatically 
recognizing humans and their movements from image 
sequences was conceived in the scope of general object 
recognition for the purposes of autonomous road 
vehicle navigation and driver support in an 
Autobahn-like environment. It essentially means an 
extension of the successful work on road following that 
emerged at UBM over the last decade; on an object 
detection level, different modules will operate on cars, 
fixed obstacles, traffic signs etc. [Dickmanns 89, 
Dickmanns 91]. The motivation for detecting human 
beings within an autopilot (and driver assistence 
system) is justified by their need of special protection 
  
in road traffic as well as their ability to signal messages 
by gestures relevant for cooperative vehicle control. 
Other a Potion of the system presented here are 
imaginable in the fields of RU and movement 
analysis [Proffitt 86], sports training or kinesiology and 
rehabilitation medicine where pathological gait 
patterns are to be analysed. 
Activities with a similar scope on the topic of visual 
body motion detection are known [Hogg 83, Hogg 88, 
Rohr Pur Various other studies have been pursued, 
e.g. emphasizing the occlusion problem, segmentation 
techniques [Leung 87] or detection by using markers. 
1.2. System concept 
Figure 1 gives an overview on the system concept. 
Instead of mere pattern matching of image information 
with previously stored shape or movement patterns the 
camera data are compared with internal assumptions 
of a spatio-temporal process model. This presumes 
motion states of an external object, i.e. a human body 
acting in a partially known environment (dynamic 
movement model). Thereby the static, unmoved state 
of a figure is treated as a special case of the figure 
being in motion. The process model maps image 
features according to its momentary internal state onto 
the image plane for comparison with data which are 
associated with the process going on in the real world. 
It also directs feature extraction after it has been itself 
initialized by image processing during bootstrapping 
including first model selection. As a figure’s effigy is 
processed : three spatial dimensions and time, the 
computer model can be regarded as a means of 
recursive reconstruction of object states occurring in 
reality, like in the observer loop concept in standard 
control theory. 
In contrast to this concept, however, a pure 
recognition task without being able to control the 
counterpart is at hand here. The process model has no 
capability to exert an influence on the outside 
obstacles it is imitating, the adjustment strategy is 
based merely on visual input. So the task of signal 
decoding, i.e. signal restoration from a known carrier 
This research project has been supported by the German Federal Ministry of Research and Technology (BMFT) and Daimler-Benz 
AG, Grant No. ITM 8900A (Prometheus PRO-ART).