If out-of-sample [generalisation] performance is more important than fast learning, then
the modular product unit neural network exhibits clear superiority. As can be seen by
comparing the ARV-values and the SRMSE-values the modular product unit neural
network model ranks best, followed by the two-stage neural network approach and the
gravity model. The average generalisation performance, measured in terms of
ARV(M3), is 0.2022 and, measured in terms of SRMSE(M3), 1.1017, compared to
0.2251 and 1.1614 in the case of the two-stage approach, and 0.2262 and 0.1658 in the
case of the gravity model15. This difference in performance between the modular
product unit neural network model and the benchmark models is statistically
significant16. If, however, the goal is to minimise execution time and a sacrifice in
generalisation accuracy is acceptable, then the standard origin constrained gravity
model is the method of choice. The gravity model outperforms the neural network
models in terms of execution time, the modular product unit network model by a factor
of 10 and the 2-stage neural network model by a factor of 102. But note that this is
mainly caused by two factors: first, that our implementations were done on a serial
platform even though the neural network models are parallelizeable, and, second, that
we implemented a rather time consuming termination criterion ( к = 40,000) to stop the
training process.
Table 3: Benchmark comparisons of the Modular Product Unit Neural Network 1CSL
with the Two-Stage Neural Network Approach Ω'''t and the Gravity
Model τ g"av for Modelling Origin Constrained Spatial Interactions
Modular Product Unit Neural Network |
Two-Stage Neural Network Approach |
Origin Constrained Gravity Model | |
In-Sample (Training) Performance ARV SRMSE |
0.2212 (0.0087) 1.2858 (0.0254) |
0.2682 (0.0222) 1.4150 (0.0578) |
0.2121 (0.0017) 1.2594 (0.0049) |
Out-of-Sample (Testing) Performance ARV SRMSE |
0.2022 (0.0150) 1.1017 (0.0407) |
0.2251 (0.0255) 1.1614 (0.0670) |
0.2262 (0.0027) 1.1658 (0.0069) |
Note: Figures represent averages taken over 60 simulations differing in the initial
parameter values randomly chosen from [-0.3, 0.3] (standard deviations in
brackets); the testing set consists of 248 patterns and the training set of 496 patterns.
28