International Archives of Photogrammetry and Remote Sensing, Vol. 32, Part 7-4-3 W6, Valladolid, Spain, 3-4 June, 1999 
135 
acquisition times of images to be interpreted. Images acquired 
simultaneously - eventually from different sensors - are 
collected in a sensor group. All sensor groups are sorted in their 
chronological order to ease the proper administration of the 
image sequence. 
During the interpretation process, the state transition diagram is 
used by a new inference rule. Analysis starts with the first image 
of the given sequence marked with time stamp tj. If a state of the 
state transition diagram can be instantiated completely, the 
temporal knowledge is used to hypothesize one or more possible 
successors of this state for the next image in the chronological 
order (time stamp £)• The system selects all successor states that 
can be reached within the elapsed time £-(/ according to the 
transition times defined in the temporal relations. States which 
are selected many times, due to loops in the transition diagram, 
are eliminated. The possible successor states are sorted by 
decreasing priority so that the most probable state is investigated 
first. All hypotheses are treated as competing alternatives 
represented in separate leaf nodes of the search tree (see chapter 
2.2.). Starting with the alternative of the highest priority, the 
hypotheses for the successor state are either verified or rejected in 
the current image. For continuous monitoring, the time stamps of 
the instances can be used to remove the old nodes of tj. 
In the example of Fig. 5, a successor state for the complete 
instance of B(tj) is determined which can be reached within a 
time step of 14 days. According to the knowledge base of Fig. 4, 
the states B, C, and D are possible. The successor B can be 
reached either via the loop B-B or via the path B-C-B, but 
identical solutions are considered only once. The node C is 
Figure 5. Search tree of the interpretation process according to the 
knowledge base in Fig. 4: Assuming a time step of 14 days 
the possible successor states of B are B, C, or D. Hence, the 
search tree splits into three leaf nodes N2(12) to N4(12). 
reached via the transition B-C and the successor D by following 
the path B-C-D omitting the intermediate state C. All three 
possibilities are treated as competing alternatives and the search 
tree splits into three leaf nodes. The system prefers the best 
judged node according to the possibilistic approach mentioned in 
chapter 2.2. If the judgements are similar the alternative of the 
highest transition priority is chosen. 
5.2. Extraction of an Industrial Fairground 
An industrial fairground is an example for a complex structure 
detectable by a multitemporal image interpretation only. Using a 
single image it would be classified as an industrial area consisting 
of a number of halls. The special use of this industrial area can not 
be detected from one time instance alone. However, during 
several weeks of the year some unnormal activity can be 
observed: exhibition booths are constructed, crowds of people 
visit the site, and the booths are dismantled again. In aerial 
images the different phases can be recognized by full or empty 
parking lots, or vehicles or people on the fairground respectively. 
This knowledge can be exploited for the automatic extraction of a 
fairground and formulated in a semantic net for a multitemporal 
image analysis (see Fig. 6). The different states of a fairground 
are represented by the concepts Fair Idle, Fair Construction, 
Fair Active, and Fair Dismantling. The construction, active and 
dismantling phase are transient compared to the state Fair Idle. 
Therefore, transition times of four to eight days are defined for 
the corresponding temporal relations. Additionally, the node 
Fair Idle is looped back to itself. 
The analysis starts with the first image of the sequence looking 
for an Industrial Area. In the given example, the system searches 
for at least three halls and one parking lot. These objects are 
represented in the image by regions of special geometric and 
radiometric properties. Halls for example are in most cases 
right-angled polygons. To verify the hypotheses suitable image 
processing algorithms are activated. Segmented regions that 
meet best the expectations are chosen, others are rejected. If the 
Industrial Area can be instantiated completely, the system tries to 
refine the interpretation by exchanging the Industrial Area by a 
more special concept. There are four possible specializations 
(Fair Idle to Fair Dismantling) and the search tree splits into four 
leaf nodes. Each hypothesis is tested in the image data. Normally 
any cars are prohibited on the fairground. But during the 
construction or dismantling phase there are trucks near the halls 
which keep the equipment for the booths. Hence, the system 
searches for small bright rectangles close to the halls. An active 
fair can be recognized by parking lots filled with cars and - if the 
image resolution is sufficient - by persons walking on the 
fairground (see images in Fig. 6). 
If one of the four states can be verified, the temporal inference is 
activated. The system switches to the next image in the sequence 
and generates hypotheses for the successor state. According to 
the elapsed time and considering the transition times all possible 
successors are determined. If for example the time step between 
the two images was two weeks, it is possible that Fair Idle 
follows immediately after Fair Active omitting the dismantling