n be 
ther 
one 
lude 
S in 
eful 
able 
the 
for 
ary 
um 
the 
the 
ons 
the 
and 
een 
atic 
the 
ide 
ons 
tial 
l'he 
in 
the 
for 
ng. 
ses 
‚of 
ard 
ny 
nd 
Ice 
he 
his 
one dimension fully agrees with the maximum information or 
entropy of the underlying stochastic process (see, e.g., Blais 
[1992b] for details). 
In two and higher dimensions, these classical Fourier based and 
parametric methods often lead to complications and ambiguities. 
More specifically, extensions of the sample autocovariance 
functions need to be compatible with causality and other 
physical requirements of the observed process. The 
nonuniqueness of the extension solution implies that the 
estimated spectrum needs to be constrained to correspond to the 
application requirements. 
Numerous researchers have investigated the use of maximum 
entropy approaches for spectrum estimation in two and higher 
dimensions. Among them are Burg [1975], Pendrell [1979], 
Wernecke and D'Addario [1977], Lim and Malik [1981], and 
Lang and McClellan [1982]. The approach of Lim and Malik 
[1981] is especially appealing with a recursive strategy using 
fast Fourier transforms and the dual of the sample 
autocovariance function. The latter has been studied and further 
discussions can be found in Blais [1992b] with a variation of 
the Lim and Malik [1981] approach having been proposed and 
experimented with in Blais and Zhou [1990 and 1992]. 
The maximum entropy approach has intrinsic features which are 
most interesting in the sense that the sample autocovariance 
function is extended in an optimal manner without using 
artificial constraints or models. The implemented conditions in 
this extension are simply the positive definiteness for 
realizability of the physical process and correlation matching for 
known lags. In other words, only the implications of the 
observed process are used in the estimation of the spectrum. 
Among the characteristics of the spectrum estimates are the 
resolution features, the consistency and reliability of the results. 
5. APPLICATIONS IN ADAPTIVE FILTER 
DESIGN 
In digital signal and array processing, filters are designed to 
restore the information by removing the noise or correcting for 
some degradation function. In several applications of signal and 
array processing, the underlying process is not stationary with 
the implication that the filters need to be adaptive to meet the 
expectations. Adaptiveness in filter design means that the filter 
parameters change whenever the conditions in the applications 
warrant it. In other words, the filters are self calibrating in their 
implementation. 
The adaptability of a mean or median filter in digital image 
processing simply implies a variable mask or template over 
which the mean and median operations are carried out. For 
instance, under smooth texture conditions, a smaller mask may 
be sufficient while under rougher conditions, the mask may 
need to be larger for reliability and other similar requirements. 
Other filter applications may have directional dependence and 
hence the detection of optimal directions may be necessary for 
adaptability to different conditions. 
The adaptability of a spectral filter such as an inverse filter 
would require a variable transfer function while an adaptable 
Wiener filter would mean a variable transfer function or spectral 
density function for the signal. In such applications, the average 
information content often plays an important role as optimal 
information extraction is the usual objective of the filtering. The 
question of deciding on an appropriate measure for the 
information content is very much dependent on the application 
context and the specific objectives of the operations. 
The problem of optimal filter design is essentially one of model 
identification strategies and information theory is well known 
for its applicability in these areas [e.g., Blais, 1987 and 1991b]. 
The observational and related information can usually be 
analyzed in terms of information contents to infer a most 
appropriate model for the application. A number of researchers 
from Kullback and Liebler [1951] to Landau [1987] and others 
have studied the use of information theory for these 
187 
applications. A number of distributional and related model 
identification results can be found in Blais [1987 and 1991b]. 
There are still several open questions in model identification 
concerning the consistency and asymptotic efficiency of the 
selected models, especially in multivariate applications and 
implementations with limited data samples and missing 
observations. Research on these and related questions is 
continuing, especially for digital image and array processing. 
6. APPLICATIONS IN INVERSE PROBLEMS 
Inverse problems are among the most common problems 
encountered in the physical sciences. With only limited 
observational and other information, inverse problems often 
present tremendous difficulties to the scientists who want 
reliable answers that are justifiable and appropriate. 
There are different classes of inverse problems depending on 
the nature of the problems and the information involved. First, 
there are mathematical inverse problems such as Cauchy 
contour integration and the inversion of integral transforms in 
purely analytical terms. Second, there are the inverse problems 
exemplified by object reconstruction and tomography which 
involve geometrical analysis as well as estimation 
considerations. Third, there are the geophysical inverse 
problems which involve much physics and geology for analysis 
and interpretation of the results. 
One implication of the preceding observations is that the study 
of inverse problems is clearly more than a simple extension and 
generalization of estimation theory. The perspective used in the 
following is that inverse problems are problems with incomplete 
information so that much of the experienced complications are 
actually due to the missing information and the implications 
thereof. One approach which has been successful in numerous 
applications is using information theory and related 
considerations. The advantages of this general approach will be 
discussed in the following with examples of applications. 
In strictly mathematical terms, the problem can be formulated as 
follows: 
u=KU 
where U describes a true state vector and u is the observed state 
vector or the perceived signal or image after having been 
corrupted or modified by mechanisms of observation or 
measurement. The direct problem is primarily one of deductive 
prediction, i.e. given prior knowledge of U and the operator K, 
deduce or estimate u. The corresponding inverse problem, i.e. 
given the observed or measured u and a specific operator K, 
estimate the true vector U. In practice, it often occurs that K is 
also poorly defined or even understood. 
The general solution to any inverse problem can be described in 
terms of Bayes' theorem which involves probabilistic measures 
of the available information. Assuming that the observed or 
measured vector u is a function of components Uy of the true 
state vector U with the probability (ulUg) known, then Bayes' 
theorem implies 
U(ulU, MU, 11) 
HU D TU CU ID 
m 
where I denotes the available prior information. In cases where 
prior information is known to be uniform, then 
WU, Iu, I) ec u(uIU, ) 
which implies a straightforward solution. 
The preceding Bayes' theorem shows how to combine partial 
information in a mathematically rigorous manner. Then the 
principle of maximum information or entropy can be used to 
arrive at optimal frequencies taking into consideration additional