FASTER TRAINING IN NONLINEAR ICA USING MISEP



1

0.5

0

-0.5

-1

-1.2     -1     -0.8    -0.6    -0.4    -0.2      0      0.2     0.4     0.6     0.8

Fig. 4. Separation performed by the MLP-based network.


0.1
0.08
0.06
0.04
0.02

0
-0.02
-0.04
-0.06
-0.08
-0.1

-0.06      -0.04      -0.02         0         0.02        0.04        0.06        0.08        0.1

Fig. 6. Mixture of a supergaussian and a subgaussian source.


1.5

1

0.5

0

-0.5

-1

-1.5

-1.5

-1

-0.5

0.5

Fig. 5. Separation performed by the RBF-based network.


1

-1

-1.5

-0.4     -0.3     -0.2     -0.1       0       0.1       0.2      0.3      0.4      0.5      0.6

Fig. 7. Separation performed by the MLP-based network.


slow training of MLP-based nonlinear ICA systems is the
nonlocal character of these networks.

Figure 9 shows an example of an ICA result obtained
with the RBF-based network, in the case of the mixture of
two supergaussians, but without weight decay. While a rel-
atively good ICA result was achieved (the estimated mutual
information is the same as in Figs. 4 and 5), the original
sources were not separated. This shows the importance of
using regularization with networks of this kind.

5. CONCLUSIONS

We have briefly presented the MISEP, a method for linear
and nonlinear ICA, which is an extension of the well known
INFOMAX method. We discussed a possible cause for the
relatively slow learning that it sometimes shows, having
conjectured that it was due to the use of non-local units in
the network that performs the ICA operation.

This conjecture was confirmed by experimental tests, in
which a system based on a radial basis function network was
compared to one based on a multilayer perceptron on the
same nonlinear ICA problems. These tests confirmed that
that system based on RBF units learns significantly faster,
and shows a lower variability of the training times. The tests
also showed, however, that the RBF-based system needs to
have explicit regularization to be able to perform nonlin-
ear source separation, contrary to what happened with the
MLP-based one.



More intriguing information

1. The bank lending channel of monetary policy: identification and estimation using Portuguese micro bank data
2. Education Research Gender, Education and Development - A Partially Annotated and Selective Bibliography
3. 09-01 "Resources, Rules and International Political Economy: The Politics of Development in the WTO"
4. Bird’s Eye View to Indonesian Mass Conflict Revisiting the Fact of Self-Organized Criticality
5. INSTITUTIONS AND PRICE TRANSMISSION IN THE VIETNAMESE HOG MARKET
6. The name is absent
7. The name is absent
8. The name is absent
9. The name is absent
10. Survey of Literature on Covered and Uncovered Interest Parities