324
The multisensorial camera: a new approach in multi-sensory pattern recognition
Robert Massen
MASSEN machine vision systems GmbH
Am Seerhein 8 , D-78467 Konstanz
tel. +49-7531-57502 , fax +49-7531-53740
Key-words: multi-sensoriai, 3D camera, colour camera, classification, segmentation
Abstract
Segmentation of a scene often suffers from the lack of information contained in every pixel. The multi-sensorial
camera produces for every pixel not a scalar attribute , but a complete feature vector with many , preferrably
uncorrelated components. We show howa ,,Colour&3D" line scan camera can be designed which generates a
feature vector ( Intensity, Hue, Saturation, Z= height) for every pixel at resolutions of typically 2048 pixels
along the line of scan and with scanning frequencies up to several kHz. The processing of such a vectorial i image
starts with a LUT-based, trainable pixel classifier who transforms the vectorial image into a stack of binary class
label images. This significant data reduction results in only little information loss and leads to further proces-
sing based on well-established binary image processing techniques.
1. Image segmentation with little robustness
One of the most common problems in vision systems is their lack of robustness against small changes in
illumination, object arrangement, shadows and specular reflections or changes in the optical characteristics of the
surface of the objects to be segmented . The poor success of symbolic image processing technologies in
industrial applications is mainly due to this lack of robustness: extraction of symbols like corners, polygones
etc. tends to fail completely, if the iconic preprocesssing such as edge detection does not perform in a proper and
reliable way. Traditionally, there have been two attempts to alleviate this uncomfortable situation:
1. use more intelligent algorithms based on more a - priori knowledge. This works well in a well-known
test scene but not so in real life where generally little reliable a - priori knowledge is available.
2. restrict to very simple pixel-based algorithms like blob analysis, projections etc. This is the day-to-day
approach for most actual vision systems.
3. work on more than just one single image, preferrably on a whole sequence of images taken under
different illuminations. This is a good approach but often fails due to the increased processing time.
We prefer to remember the good old rule of pattern recognition :
Do not try to squeeze out the last drop of information from a single feature but find additional,
uncorrelated features.
We like to translate this into : Get more information from a pixel than just grey-level intensity.
2. À feature vector attached to every pixel
With the introduction of colour, we jump from a scalar pixel to a vectorial one: every pixel carries now
information on intensity and saturation and hue . Of course we should be careful about the RGB colour vector,
whose components are highly correlated and which is not a good colour space for segmentation and
classification. Our Colour Brain family of vision systems which uses colour matrix- or line cameras and
trainable, look-up table based classifiers / 1, 2/ has shown how the segmentation of a scene is dramatically
improved due to the increased information content in every IHS colour pixel compared to a dumb greylevel
pixel.
IAPRS, Vol. 30, Part 5W1, ISPRS Intercommission Workshop “From Pixels to Sequences”, Zurich, March 22-24 1995