X + 
Figure 1. Concept of vehicle manoeuvres recognition problem 
Once objects are extracted, tracking is accomplished by point- 
to-point, edge-to-edge, or area-to-area matching. These 
approaches, however, have limitation, because enough 
information cannot be derived from single image. 
The recognition of objects not only in each image, but also in 
successive images is more plausible for the vehicle manoeuvres 
recognition. The vehicle  manoeuvres recognition is 
accomplished in spatio-temporal images. In other words, 
problem of interest is identified as forming the region of 
vehicles in spatio-temporal images. Figure 1 shows the concept 
of vehicle manoeuvres recognition problem. In this sense, 
vehicle recognition includes vehicle extraction and vehicle 
tracking. 
2.2 Vehicle Manoeuvres Recognition Method 
Considering the human perception, we will develop a vehicle 
manoeuvres recognition method. Visual information has three 
features, that are spatial feature, temporal feature and colour 
feature. In order to recognize moving objects, spatial-temporal 
feature and colour-temporal feature take a important role. 
Spatial-temporal feature and colour-temporal feature correspond 
to background subtraction and optical flow extraction in image 
processing, respectively. Employing these two features, the 
pixels that are adjacent and have similar features are grouped. 
Consequently, vehicle clusters are formed in the spatio-temporal 
image. We call these process as spatio-temporal clustering 
method. Figure 2 shows the flow of spatio-temporal clustering 
method. Next section will discuss details of the proposed 
method. 
3. SPATIO-TEMPORAL CLUSTERING METHOD 
3.1 Geometric Correction 
Although laborious, image sequence must be aligned first of all. 
We develop an automatic geometric correction for vehicle 
recognition. The most popular method for geometric correction 
is to employ Ground Control Points (GCPs). Geometric 
correction can be summarized as follows: 
(1) GCPs are specified as reference points which should be 
clearly perceived and whose coordinates should be known; 
(2) GCPs determine a coordinate transformation by the least 
squares method. 
In this study, we set corners of edge contours of the markers on 
the road such as crossings and speed indicators as GCPs. Since 
the markers can be restricted in the sense of colour and shape, 
  
| Geometric Correction | 
  
background 
image 
  
  
Y 
i background subtraction 1] 
  
  
| shadow detection 
| 
| optical flow extraction 
i 
| spatio-temporal clustering | 
  
  
features 
e Bptical flow 
e Background subtraction 
value 
  
  
| vehicle recognition | 
  
Figure 2. Flow of vehicle manoeuvres recognition 
SUSAN operator (Smith and Brady, 1997) restricted in colour is 
applied to edge detection, and then the edge contours are 
eliminated by shape constraints, that are aspect ratio, number of 
pixels, and whether closed or not. 
Because extracted GCPs are the points of static area basically, 
displacement vectors of the GCPs between adjacent images are 
very small. Therefore the searching window to find 
corresponding points can be limited only within a few pixels 
square. 
Using the specified GCPs, a coordinate transformation is 
determined. In general, Affine transformation or projective 
transformation are selected as coordinate transformation. The 
transformations assume that the scene is a planer surface 
without depth variation. This assumption can be available when 
and only when the field-of-view (FOV) of cameras is enough 
narrow (typically 5 degree or less) (Irani and Anandan, 1998). 
Moreover, the coordinate transformations should coincide GCPs, 
which are the corner of markers on the roads, because the 
markers affect results of following background subtraction. We 
utilize Kriging interpolation as geometric correction so that the 
GCPs can be coincided. Kriging denotes a body of techniques 
to predict data at arbitrary locations with some observations 
—562— 
  
reco 
cove 
3.2 
Bacl 
pixe 
imag 
time 
cons 
cann 
Ther 
back 
illun 
algoi 
A si 
diffe 
diffe 
vehi 
we € 
space 
(a) 
(b) 
(c) 
Thro 
RGB 
back 
wher 
imag 
Shad 
regio 
will a 
vehic 
neces 
A sh: 
be ar 
algor 
I(x, y 
where 
othen 
colou 
(x, y)