International Archives of Photogrammetry and Remote Sensing, Vol. 32, Part 7-4-3 W6, Valladolid, Spain, 3-4 June, 1999 
into straight line segments and arcs (curvilinear segmentation, 
obtained with the ? -s method), straight lines are grouped to 
polylines, and polylines with 90 degree vertices are singled 
out as they play an important role in many man-made objects. 
Similarly, surface patches within a hump, that can be 
analytically approximated, are found with various methods. A 
particularly efficient method is the Hough transform for 
finding planar surface patches. Here, the parameters of planes 
defined by the triangles that have been determined by a 
Delauney triangulation, enter the accumulator array. A cluster 
analysis of the accumulator array indicates triangles that lie in 
the same plane. As an optional step, all points found to be in a 
plane are used in a least-squares adjustment for determining 
the parameters more precisely. Again, the motivation for this 
step is related to the observation that man-made objects 
consist of planes that have certain preferred orientations. 
5. CONCLUDING REMARKS 
We have described a conceptual multisensor fusion system for 
the purpose of recognizing objects of urban scenes. The 
system is characterized by a multi-stage fusion approach that 
includes laser scanning data, aerial and multispectral imagery. 
Experimental results confirm the proposed architecture 
(Figure 6). The experiments are performed with the test 
dataset of Ocean City. The test site is a rather complex urban 
area, including residential and commercial areas with 
buildings of different size and shape, live and dead 
vegetation, and trees right next to buildings. 
In light of an automatic system, we performed unsupervised 
classification of the multispectral data with the number of 
classes and the band selection/combination as parameters. 
The classification results are remarkably robust. That is, the 
parameter selection does not appear to be very crucial. The 
results also clearly demonstrate the usefulness of multispectral 
data for object recognition. Moreover, they confirm that 
object recognition in complex urban scenes cannot be reliably 
solved with one sensor only. 
Most fusion processes are carried out in the 3-D object space. 
This requires that for every sensor a relationship between the 
sensor space and object space is established. Another 
interesting challenge for fusion is related to the problem that 
sensors have different resolutions. For example, aerial 
imagery and multispectral imagery of the test site differ in 
resolution by a factor of ten. We solve this problem with a 
scale space approach. 
Future research will address the problem of object modeling, 
taking into account the features that can be extracted from 
different sensors. By the same token, more research will be 
devoted to inference processes and identity fusion. 
REFERENCES 
Abidi, M. A., and R. C. Gonzalez, 1992. Data Fusion in 
Robotics and Machine Intelligence. Academic Press, Inc., San 
Diego, CA, 546 p. 
Csathd, B.M., and T. Schenk, 1998. A multisensor data set of 
an urban and coastal scene. Int’l Arch, of Photogr. and Rem. 
Sensing, Vol. 32, Part 3/1, pp. 588-592. 
Csathd, B.M., W.B. Krabill, J. Lucas, T. Schenk, 1998. A 
multisensor data set of an urban and coastal scene. Int’l Arch, 
of Photogr. and Rem. Sensing, Vol. 32, Part 3/2, pp. 588-592. 
Githuku, A., 1998. A conceptual framework for object 
recognition and surface reconstruction from high resolution 
multispectral and hyperspectral imagery. Int’l Arch, of 
Photogr. and Rem. Sensing, Vol. 32, Part 3/1, pp. 452-463. 
Hall, D., 1992. Mathematical Techniques in Multisensor Data 
Fusion. Artech House, Boston, London, 301 p. 
Jaw, J., 1999. Control Surfaces in Aerial Triangulation. Ph.D. 
dissertation, Department of Civil and Environmental 
Engineering and Geodetic Science, The Ohio State 
University. 
Josephson, J., and S. Josephson, 1994. Abductive Inference. 
Cambridge University Press, 306 p. 
Krabill, W. B., R. Thomas, K. Kuivinen, and S. Manizade, 
1995. Greenland ice thickness changes measured by laser 
altimetry. Geophysical Research Letters, Vol. 22, No. 17, pp. 
2341-2344. 
Krupnik, A., 1996. Using theoretical intensity values as 
unknowns in multiple-patch least-squares matching. 
Photogrammetric Engineering and Remote Sensing, 
62(10): 1151-1155. 
Nadler, M., and E. Smith, 1993. Pattern recognition 
engineering. John Wiley & Sons, New York, pp. 299-302. 
Schenk, T., 1995. A layered abduction model of building 
recognition. In: Grim A., Kuebler O., Agouris P. (Eds.), 
Automatic Extraction of Man-Made Objects from Aerial and 
Space Images, Birkhauser Verlag, Basel, pp. 117-123. 
Schowengerdt, R. A., 1997. Remote Sensing, Models and 
Methods for Image Porcessing. Academic Press, San Diego, 
522 p. 
Shufelt, J. A., 1999. Performance evaluation and analysis of 
monocular building extraction from aerial imagery. IEEE 
Transactions on Pattern Analysis and Machine Intelligence, 
21(4), pp. 311-326.