Situatlon assessment 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
Objekimodule 
| Objektmanager ie 
Road other 
module | | Objekt 
modules 
Simult. Initlall- Objekt 
Shape Salon Jack ng 
+ or 
bjekt tat 
ES cette. | [esting 
| Feature extraction 
| 
Video data 
Figure 2 Structure of the object recognition module 
An already existing software module performs the de- 
tection and the tracking in the image plane of an single 
not occluded object Ot’ driving in front of the own 
vehicle [Solder, Graefe 90], [Regensburger, Graefe 90] 
and [Thomanek, Dickmanns 92]. Algorithms on a par- 
allel processor system exploiting high level spatio-tem- 
poral object models estimate recursively in real-time 
the relative state of the object using Kalman filtering 
techniques for state estimation [Dickmanns, Christians 
89] and [Dickmanns et al. 90]. Yet another module 
determines the shape of the tracked object Ot’ while 
the aspect conditions change during motion [Schick, 
Dickmanns 91]. These informations are the input to an 
extended object recognition module treating more 
complex situations where occluded objects may appear, 
for example, while lane changing, If an additional, par- 
tially occluded, object has been detected and verified, a 
module like the one mentioned above is initialized to 
track the new object. The approach presented in this 
paper allows the handling of occluded 3D rigid objects 
on curved roads in real-time image sequences by com- 
bining knowledge based methods for feature matching 
and motion classification with techniques from system 
theory for motion estimation. 
This vision system uses a-priori knowledge about the 
expected objects concerning possible shape, position, 
and motion to generate a hypothesis of an object in the 
scene. The knowledge base contains the modeling in- 
formation about different object shapes (e.g. car types) 
with their characteristic features and about constraints 
in motion. An internal representation in the computer 
system of the world around the camera, installed in the 
vehicle, is required in the analysis by synthesis'-method 
selected because the system compares a generic inter- 
nal model with the real situation outside. Therefore, 
different coordinate systems have to be introduced (see 
Section III). The first hypothesis instantiated by the 
real-time object tracking module is a 3D-rectangular 
parallelepiped (wire frame model) encasing the object. 
Another, more sophisticated process estimates the 
shape in more details, but up to now not under real-time 
conditions. The tracking process evaluates a set of state 
variables containing the object position and motion by 
using a 4D-model. A further process estimating the 
curvature of the road communicates its results to the 
object tracking module, which is then able to determine 
the relative position of object ’Ot on the road. 
III. MODELING OF THE PERCEIVED SCENE 
The internal representation of the real world consists 
mainly of two parts. First it is necessary to introduce a 
different coordinate system (x,y,z,p,0,p) for each object 
in the real world, e.g. ego-car, camera, road, other cars, 
traffic signs, ..., to allow modeling of independent move- 
ments between the different objects in the scene (figure 
3). This implies for the object recognition task in an 
autonomous road vehicle guidance application an extra 
coordinate system P for the two axis platform, which 
allows to control the pan and the tilt angle of the camera 
in order to track an object in the environment. The other 
coordinates (x,y,z) are constant relative to the ego- 
car. The main coordinate system E belongs to the own 
vehicle, which is adjusted every system cycle of the 
image processing system, because every object location 
or motion is only relevant relative to the own car's 
position. A separate road and ego-state module deter- 
mines the state variables of the ego relative to the road 
and of the curvature describing the road in front of the 
car (in the future also in the back ) in the viewing range. 
A further coordinate system R has to be introduced for 
the description of the road, because the estimated posi- 
tion of the other cars moving on the road are only 
relevant relative to the road. To describe the curvature 
of the road a clothoidial representation is applied. Fi- 
nally, there exists a specific coordinate system O for 
each object driving on the road. The distance of other 
cars to the own is approximated by the Pythagoras 
equation of the road coordinates at the estimated loca- 
tion of the tracked vehicle. This technique is well suited 
for slight curvatures. That way the motion of an tracked 
object can be transformed in camera fixed coordinates 
K by calculating 
  
  
  
  
  
  
  
  
Figure 3 
Coordinate systems for modeling