International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B5, 2012 
XXII ISPRS Congress, 25 August - 01 September 2012, Melbourne, Australia 
341 
available properties of the object (spatial information, appear 
ance, and features) are used to compute a distance. When cal 
culating the spatial distance, we estimate the position of the pre 
viously detected object in the current frame by using a Kalman 
filter. The Kalman filter not only allows us to predict, but also to 
smooth the trajectory. 
Using the computed distance we can estimate how likely the de 
tected segments represent the already tracked object. All likely 
matches are combined and their position, appearance, and fea 
tures updated. Additionally we check the features in every object 
for both inconsistent movements between frames and duplicate 
features that probably represent the same area. 
Classification and Person Detection Although the described 
system can handle all kinds of moving objects, we focus on per 
sons and therefore implemented a method to differentiate be 
tween persons and other objects, like cars or dogs. We used a 
person detector, which utilizes Histogram of Gradients and Sup 
port Vector Machines (Dalai and Triggs, 2005) to detect persons 
in the previously generated segments (Section 3). 
By including this information, we can differentiate single persons 
in a group that would otherwise have been treated as a single big 
object. 
Handling of Occlusions Our algorithm uses different approaches 
to tackle partial and complete occlusions. In case of a partial oc 
clusion no segment in the current frame will match with the pre 
viously detected objects in the basic tracking step. But some of 
the object’s saved features will match with the features of the vis 
ible part of the object. Here we compute the average movement 
of these features and use this movement to adjust the position of 
the object. Therefore we can track partly occluded objects, as 
long as some of the object features match in this frame. Espe 
cially in crowded scenes, where segmentation of single persons 
is difficult, the tracking in subsequent frames can benefit from the 
use of features. 
In the case of a complete occlusion another approach is needed 
(Figure 3). First of all, we use Kalman filters to predict the posi 
tion of moving objects. If a person continues its movement, it is 
often correctly matched after it reappears. But unlike cars, per 
sons are highly mobile and can change their directions and speed 
very fast. Therefore, persons can appear at various locations after 
disappearing behind an obstacle, and so matching it with the help 
of the predicted position is not possible. Additionally, objects 
often reappear partly occluded and a matching with the original 
object fails. Instead of matching it to the previously tracked ob 
ject, a new object is created and tracked subsequently. As the 
object fully emerges from the occlusion, the object’s appearance 
eventually looks like the previously tracked object. This leads to 
the situation that one real object can have two representations in 
the tracker. This error can be detected by computing a distance 
between the two objects using all properties of the objects (spa 
tial information, appearance, and features). If it is plausible that 
the two objects in the tracker represent the same real object, the 
objects in the tracker are fused and their information is combined. 
Scene Information During the tracking of objects and persons, 
we also store information about the observed scene in order to 
improve tracking results in subsequent frames. These properties 
include the common direction and speed of objects at all posi 
tions as well as entry and exit zones. 
The gathered scene information is used in various parts of the 
tracking algorithm. The direction and speed can be used in the 
motion model of the Kalman filters to support the prediction of 
Figure 3: Top left: Tracked person (red) approaching occlusion. 
Top right: Person hidden by occlusion, tracker searches for per 
son at the predicted position (red dashed). Bottom left: Person 
emerges partially at an unpredicted position. A new tracked per 
son is created (blue). Person emerges fully, is matched with both 
red and blue tracked persons. The data is fused and stored for the 
next frames. 
the position. The entry and exit zones can help to support the 
matching in case of occlusions. 
These patterns also allow us to detect any unusual behavior that 
might indicate a dangerous situation and requires special atten 
tion. Examples of such behavior would be a sudden change of 
preferred routes, which might indicate a panic or a blocked path 
or doorway. 
An example of generated entry zones can be seen in figure 4. The 
zones were generated over 22000 frames with 166 used persons. 
Among some smaller erroneous entry zones along the person’s 
path, larger and intense entry zones at the border of the image 
can be seen. Additionally, a few obstacles (e.g. trees) are sur 
rounded by entry zones, as the occlusions can lead to temporarily 
lost tracking. 
The corresponding exit zones are similar to the entry zones. Both 
passing behind an occlusion and entering and leaving the field of 
view produces both entry- and exit points. 
Figure 4: Example of generated entry zones within 22000 frames 
with 166 used persons.