GENERALIZATION TECHNIQUES FOR LAYERED NEURAL NETWORKS
IN THE CLASSIFICATION OF REMOTELY SENSED IMAGES
Eihan SHIMIZU and Morito TSUTSUMI
Department of Civil Engineering
University of Tokyo
Japan
shimizu@planner.t.u-tokyo.ac.jp , tsutsumi@planner.t.u-tokyo.ac.jp
Le Van TRUNG
Department of Civil Engineering
The National University of Hochiminh City
Vietnam
trung.lv@ut-hcmc.cinetvn.com
KEY WORDS: Layered neural network, generalization, Akaike’s Information Criterion, Tikhonov’s regularization.
ABSTRACT
In recent years, researchers have paid a lot of attention to Layered Neural Networks (LNNs) as a non-parametric approach for the classi
fication of remotely sensed images. This paper focuses on the generalization capability of LNNs, that is, how well an LNN performs with
unknown data. First, we clarify its description from the point of view of information statistics. With this discussion, we provide a fea
sible technique to design the LNN in consideration of its generalization capability. Finally, we apply the proposed technique to a practi
cal land cover classification using remotely sensed images, and demonstrate its potential.
1 INTRODUCTION
Among supervised classification methods for remotely sensed
data, Maximum Likelihood Classification (MLC) is presently the
most widely known and utilized (e.g. Curran and Hay (1986),
Yool et al. (1986)). MLC is often used as a standard classification
routine against which other classification algorithms are com
pared. This popularity is due to the fact that MLC is the optimal
classifier in the sense of minimizing Bayesian error. However,
MLC is a parametric classification method where the underlying
probability density function must be assumed a priori. We may
obtain a poor MLC performance if the true probability density
function is different from what is assumed in the model. In recent
years, researchers have attempted to provide non-parametric
classification methods to overcome this disadvantage of MLC,
and Layered Neural Networks (LNNs) have been proposed as
suitable for the efficient classification of remotely sensed images.
When using an LNN classifier, however, users have often been
faced with a generalization problem; generalization is concerned
with how well an LNN model performs with input on which the
model has not been previously trained. That is, an LNN classifier
usually performs well on a set of training data, but it may not
guarantee good generalization over all unknown data during the
actual classification process.
This paper discusses LNN classifier generalization, a controversi
al and often vague term in the neural network literature (Wan
(1990)), and clarifies its description. We introduce some tech
niques for generalization based on Akaike’s Information Criterion
and provide a feasible technique to design an LNN in considera
tion of its generalization capability. Finally, we apply the pro
posed technique to a practical land cover classification using
remotely sensed images, and demonstrate its potential.
2 BASIC FORMULATION OF THE LAYERED NEURAL
NETWORK CLASSIFIER
It has been proved that a three-layered neural network, when the
appropriate number of nodes are set in the hidden layer and the
sigmoidal activation function is used in the hidden and output
nodes, can approximate any continuous mapping (Gallant and
White (1988) and Funahashi (1989)). Therefore, in this study, we
only focus on three-layered neural networks, as shown in Figure
1.
Input Layer Hidden Layer Output Layer
Figure 1 Architecture of three-layered neural network.
PI-2-1
