43 PN AD-u
phase. The training set, once again, contained the
complete set of eleven control points. The curves
therefore reflect the manner in which the two networks
learn the function.
The two curves in Figure 8 are clearly distinguishable,
each possessing different features peculiar to their
method of learning. The stand-alone network, after 1000
iterations, has a geo-referencing error of 2000 m. For
clarity, this is off the figure in order to show the lesser
undulations within the learning curve. From the geo-
referencing error of 2000 m the stand-alone network is
quick to learn the hidden patterns within the geo-
referencing function and swiftly progresses to errors of
around 150 - 200 m level after 2000 iterations. The slight
perturbations around the 150 m level are a feature of the
noise added to the network to avoid the occurrence of the
network falling into a false well.
The hybrid network curve within the figure shows a much
smoother learning path in contrast to the stand-alone
network. This curve has a geo-referencing error of 1150
m at 1000 iterations, again for clarity, this is off the graph.
This initial value is almost half that of the other network.
However, using the hybrid approach the network fails to
learn the function at such a rapid rate. The learning curve
is gradual and only approaches a stable value of about
50 m at 25 000 iterations (taking over 10 times as long as
the stand-alone model to stabilse). Though the two
graphs cross at approximately 8000 iterations the
remaining 17 000 iterations have the effect of gradually
reducing the geo-referencing error in the hybrid network
which ultimately results in a final geo-referencing error 2.5
times smaller than that of the stand-alone model.
5.2 Benefits of Hybrid Networks over stand-alone
Networks
The final test was to try to achieve similar geo-referencing
precisions to the hybrid network using a stand-alone
neural network model. The only variable to be altered in
the tests was the number of hidden processing units
within the single hidden layer. The learning term and the
momentum rate were kept at the constant values of 0.15
and 0.6 for all stand-alone neural network topologies and
0.1 and 0.5 for all hybrid network topologies. This was
necessary to keep the number of tests to a realistic
amount. Tests were performed using between 2 and 44
hidden layer units. Some typical results from the tests
have been selected and presented in Figure 9. The tests
were performed to the exhaustive limit of 100000
iterations.
Some of the statistics from the tests are presented in
Table 2. From Figure 9 and Table 2 the noise within the
learning process can be quantified by examining the
standard deviation (std dev) of the scatter from the stable
region of the curve. Despite the network achieving
817
precisions of the order of 70 m (for 20 hidden units) the
standard deviation of 12.3 m indicates quite large
deviations from a smooth learning curve. This quantity of
noise is also present on the remaining two curves (those
for 6 and 10 hidden units). The curves suggest the stand-
alone neural network is capable of producing comparable
results to the hybrid network (i.e., 70 m level of precision).
However to achieve this, the network requires additional
processing units within the hidden layer (e.g., from Table
3 the number of units required is 20) and hence additional
computational time to learn the function, even then there
is a large uncertainty, i.e, 12.3 m, associated with the
geo-referencing precision.
Number of Hidden Layer Units
vores: Be iil 20
= 290
5 180
= — 160
= 140
$ 120 a, A 1 yu ts
c 100 Peg NOI ez,
S 80
© 60
3 40
à 20
04
O 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000
Number of Iterations
Figure 9 Geo-referencing Error against number of
iterations for different Stand-Alone Network
Topologies
No of Hidden Stand-Alone Network
Units mean (m) std dev (m)
6 153 12.1
10 107 11.4
20 70 12.3
Table 3 Statistics of the Learning Curves
in Figure 9
Similar tests were performed for the hybrid network to see
if prolonged training would lead to improved geo-
referencing. The results are presented in Figure 10. The
most noticeable feature is that all three curves produce
similar geo-referencing errors.
Figure 10 demonstrates that when using a hybrid network
the final geo-referencing error is less dependant upon the
topology of the neural network. However, as can be seen
within Table 4, the final result may be similar for all hybrid
network configurations (i.e., 6, 10 or 20 hidden units) but
the noise in the learning curve gets progressively worse
the more redundant hidden layer processing units the
network possesses.
International Archives of Photogrammetry and Remote Sensing. Vol. XXXI, Part B4. Vienna 1996
