10 
realization of the well known Canny edge detector [43]. Measured deviations of the observed PSF compared to the ideal PSF are 
only about 1% of the maximum filter value. 
Many other smart image sensors with varying capabilities have been demonstrated and described in the literature. They exhibit 
varying degrees of on-chip functionality or “smartness”. This smartness can also consist of the duplication of some understood 
functionality of biological vision systems [44], leading to a special type of smart image sensor, grandly termed “seeing chips” [3]. 
In that sense, the convolution CCD summarized above, with its dynamic convolution capability, might also be called a seeing chip: 
Not only is it known that the convolution is - under certain assumptions - an optimal object recognition filter [45], there are also 
indications that human beings localize noisy patterns with a similar matched filtering mechanism [46]. Still, it is considered 
premature to call such developments seeing chips but the motivation and the potential are clearly there. What can be expected, 
therefore, from this very active area of research ? 
7. SEEING CHIPS 
One school of thought argues that the best approach to seeing chips is to learn from nature and to emulate her strategies in silicon, 
step after step, by using low-power analog circuitry [47]. With the functionality of biological vision systems, it is expected that also 
the visual sense can be realized on a so-called silicon retina. In many of these approaches, the automatic gain control of biological 
vision systems in the acquisition of images is emulated, resulting in non-linear, locally varying and scene-dependent sensitivity 
curves, often out of external control [47]. As a consequence, it seems difficult in this approach to unite visual perceptive functio- 
nality with exact radiometric behavior as required for metrological applications such as digital photogrammetry and dimension 
control in automatic manufacturing. Silicon retinas for a variety of tasks have been demonstrated, including the detection of optical 
flow fields, motion estimation, edge or discontinuity detection, point/linear/area ranging and stereo vision, time-to-contact 
determination, symmetry detection, lateral and rotational object localization based on moments, figure-ground segregation, 
calculation of spatio-temporal derivatives, and other vision tasks, see for example Reference [3]. As interesting and impressive 
these demonstrations may be, their extendability is lacking tremendously in one aspect, where biology has an invaluable advantage: 
3D integration. Nature’s processing systems are essentially 3D, capable of stacking “processing planes” in the volume of biological 
material. In contrast, silicon technology is really only two-dimensional. This means that adding processing power at a pixel site 
will automatically increase the necessary floor space and reduce the pixel’s fill factor (photosensitive to insensitive area). As a 
consequence, recent developments in the area of integrated machine vision also consider architectures based on different planes: an 
image acquisition plane might be followed by several (analog) preprocessing planes, an (essentially digital) classification plane and 
an output plane, all connected using suitable high-bandwidth bus schemes with an appropriate software protocol. This guarantees a 
maximum fill factor for the image sensing part and allows to use optimal architectures and technologies for the different parts of 
the complete system. Such an approach does not necessarily mean that every plane resides on its own chip; different planes can be 
integrated on the same chip, as envisaged for example in the feature extractor imager described in [48]. The technology for stacking 
and interconnecting silicon chips, so called 3D or z-plane technology, has been developed [49], but the appealing idea of a low-cost 
single-chip vision system, a seeing chip, becomes seriously compromised. 
The conclusion is that smart image sensors - offering additional on-chip functionality - and integrated vision systems are certainly 
trends that will lead to a wide range of practical products, albeit rarely in the form of single, self-contained seeing chips. It can 
rather be expected that smart image sensors with extended capabilities for the dynamic acquisition of images will be part of an 
integrated vision system, consisting of an economically sensible combination of imager, analog and digital processing parts. Special 
properties built into such smart image sensors include lower noise, higher D/R, programmable sensitivity, on-chip non-uniformity 
and shading correction, variable exposure and timing control, region-of-interest capability, dynamic pixel size and shape, on-chip 
image pre-processing which can be carried out for all pixels in parallel, etc. It might well be that seeing chip is a misnomer, and 
that the silicon retina - raising less exaggerated expectations and suggesting more of a front-end image acquisition/pre-processing 
system - is a much more appropriate name for the current and future sensing device developments in electronic imaging. 
8. CONCLUSIONS AND DISCUSSION 
The wide field of electronic imaging seems to include two quite distinct sub-fields of integrated imaging of visible scenes, 
characterized by the motivation of what to do with optically imaged scenes: On one hand, there are the applications where a 
representation of the real world is sought, that is as accurate as possible, either for reproductions sake (for example in electronic 
photography or video applications) or for the extraction of meaningful geometric or radiometric quantities (for example in digital 
photogrammetry or in many optical measurement techniques for industrial applications). On the other hand, there are machine 
vision applications, concerned with the interpretation of the scene, the perception of what the different objects are in a picture. The 
prototypical model for these applications is, of course, the human visual system with its amazingly robust and versatile recognition 
capabilities. It is not surprising that these two sub-fields pose varying requirements on the image sensor: In the first case, the 
requirements are in terms of increased pixel numbers, dynamic range, readout speed, geometric and radiometric accuracy 
(linearity), pixel registration, etc. In the second case, functionality and robustness are of paramount importance, since the output of 
the complete sensing and interpretation process is only a few bytes, in extreme cases just one bit (pass or fail, present or absent, 
danger or normality, etc.). The performance in many other aspects can be sacrificed for that; as an example, the pixel number could 
be traded in for increased processing power and functionality in a pixel. The analysis presented in this paper demonstrates that 
recent developments, as well as current research, is producing new devices and concepts for the two above mentioned sub-fields of 
electronic imaging: 
IAPRS, Vol. 30, Part 5W1, ISPRS Intercommission Workshop "From Pixels to Sequences”, Zurich, March 22-24 1995 
abc 
fev 
eff 
Re: 
the 
for 
pix 
On 
[1] 
Cou 
[2] ! 
Gra 
[3] 
[4] : 
[5] ] 
[6] ] 
[7]: 
[8] 1 
[9] | 
Pro 
[10] 
[11] 
[12] 
IEE 
IAP