ET TEEN NS EEE NE TENTE 
FROM PIXELS TO ANSWERS — Tod 
RECENT DEVELOPMENTS AND TRENDS IN ELECTRONIC IMAGING phoi 
ima 
P. Seitz, O. Vietze and T. Spirig adv. 
Paul Scherrer Institute Zurich, Badenerstrasse 569, CH-8048 Ziirich : mos 
2 Phone: +41-1-492-6450 . FAX: +41-1-491-0007 feat 
Email: seitz@cvax.psi.ch , vietze @cvax.psi.ch ; thomas.spirig@cvax.psi.ch desi 
evo 
yeaı 
KEY WORDS: Electronic Imaging, CCD, Image Sensors, Cameras, Seeing Chips emr 
fabr 
for 
SUMMARY sam 
fabr 
Electronic imaging is the emerging interdisciplinary science of studying, designing and realizing complete, optimized imaging Stan 
systems as holistic entities. Progress is reported from different fields of electronic imaging: The number of pixels on an image emr 
sensor is steadily increasing; Skx5k pixels are commercially available today. At the same time, the geometry of the pixels is todz 
reduced; today's smallest pixels measure only 5.1x5.1 um?. An unprecedented dynamic range of 94dB has been obtained in a CCD larg 
video image sensor at room temperature, with an equivalent r.m.s. readout noise of less than one photo-electron. Single-chip whi 
cameras with on-chip analog-to-digital converters for less than $10 are advertised. Image sensors with on-chip functionality are us 
being developed, such as real-time selectable pixel size and shape, the capability of performing arbitrary convolutions rule 
simultaneously with the exposure, variable sensitivity of the pixels, or the synchronous detection of 2D modulated light fields for sens 
real-time range imaging. It is concluded that the holistic approach of electronic imaging will influence the design and the as u 
performance of future imaging systems in many disciplines, reaching from digital photogrammetry to machine vision on the factory rule 
floor and in robotics applications. The 
incr 
redi 
1. INTRODUCTION : fabr 
Solid state image sensing has made enormous progress since the first demonstration of an image sensor using the charge-coupled equ 
device (CCD) principle in 1971. At that time, the first image taken with a CCD line sensor exhibiting 96 pixels was published [1]. pr 
Only 21 years later, in 1992, the successful fabrication of a CCD image sensor with more than 26 million (5120x5120) pixels was fedi 
reported [2]. This impressive development was only possible because of the amazing progress made by silicon semiconductor cust 
technology, largely driven by the ever-increasing demand for digital computer memories with higher and higher storage capacity. In it px 
fact, it is not only with larger number of pixels that image sensors have profited from the advances in semiconductor technology. larg 
On-chip functionality has increased as well, and first successes with image sensors carrying out image processing algorithms on the aren 
sensor chip has led researchers to declare the imminence of powerful "seeing chips” [3]. In any case, the image sensor is just a proc 
component, albeit a central one, in a complete optical information acquisition and processing system. The study, design, realization man 
and characterization of such systems is the topic of the emerging discipline of electronic imaging [4]. Electronic imaging is not only are 
relying on solid state technology for the acquisition of images and the integration of subsequent image processing functions. men 
Significant research activity is reported, among other things, in making the optical imaging process — traditionally carried out by soit 
glass lens systems — dynamic, through the use of electrically controllable liquid crystal material, often in combination with fabr 
diffractive optical devices, see for example Reference [5]. spar 
The purpose of this paper is to review recent progress in two of the most relevant fields of electronic imaging: Large-area CCD deve 
image sensors for high-fidelity image acquisition and "smart" image sensors with on-chip functionality for machine vision. To canr 
achieve this goal, the paper is organized in the following way: In Section 2 capabilities, trends and boundary conditions of modern in | 
silicon technology for image sensing are discussed. The consequences of this in the fabrication of image sensors with more and com 
smaller pixels are presented in Section 3. It is also shown that the optical diffraction of the imaging lens puts a lower limit to the from 
pixel size, ending the trend of making pixels increasingly smaller. In Section 4, the physical limitations of the performance of picti 
image sensors are explored. It is concluded that modern image sensing techniques already come close to the fundamental noise the c 
limit imposed by the Poisson noise of light [6]. Present and future applications of CCD image sensors in electronic photography are few 
discussed in Section 5. While professional applications are in the middle of the process of "turning digital", consumer applications 
have only just started to replace the conventional photographic film with solid state image sensors and storage media. Image 
sensors with increased on-chip functionality are reported in Section 6. Whether such image sensors can already be called "seeing 
chips" is discussed in Section 7, together with the implications for machine vision. A short summary and conclusions are offered in 
the final Section 8. For 
This 
2. MODERN SEMICONDUCTOR TECHNOLOGY FOR IMAGE SENSING f «o 
avai 
Silicon is by far the predominant material in modern semiconductor technology for analog and digital processing circuitry [7]. It is clear 
very fortunate, therefore, that silicon is also extremely well suited for the detection of visible and near infrared light. As shown in a vic 
Fig. 1, the spectral sensitivity of a good silicon CCD image sensor exhibits quantum efficiencies of between 55% and 72% in the mod 
spectral range that is also accessible to the human eye. 
IAPI 
IAPRS, Vol. 30, Part 5W1, ISPRS Intercommission Workshop "From Pixels to Sequences", Zurich, March 22-24 1995