Accurate, fast and stable denoising source separation algorithms



which is identical to the one used in FastICA. Here J(s) is the Jacobian of f(s).
Interpreting the speedup as a spectral shift corresponding to Gaussian noise
gives an intuitive explanation to why FastICA is able to extract both super- and
sub-Gaussian signals with the same nonlinearity: power-method-like iterations
converge to the eigenvector whose eigenvalue has the largest magnitude. The
sign of the eigenvalue is different depending on whether the component is super-
or sub-Gaussian but the magnitude increases when moving away from Gaussian
signal whose eigenvalue has been shifted to zero.

In general, iterations converge faster with the FastICA-type spectral shift (8)
than with the global Gaussian approximation (7) but the latter has the benefit
that no gradients need to be computed. This is important when the denoising is
defined by a complex nonlinear procedure such as median filtering.

Neither of the spectral shifts, (7) or (8), always results in stable or fast con-
vergence. Sometimes the spectral shift is too large, which due to the nonlinear-
ity of denoising typically leads to oscillatory behaviour: the iteration oscillates
between two weight values. Some other times the spectral shift is too modest
leading to slow convergence characterised by small changes of
w in the same
direction during several iterations.

For this reason, we have suggested a simple stabilisation rule [1]: instead of
updating
w into wnew defined by (5), it is updated into

wadapted = orth(w + γ∆w)                     (9)

w = wnew - w ,                      (10)

where γ is the step size. Originally γ = 1, but if the consecutive steps are taken
in nearly opposite directions, i.e., the angle between
w and wold is greater
than 179
o, then γ = 0.5 for the rest of the iterations. There exist a stabilised
version of FastICA as well [2] and a similar procedure has been used in practice.

The above modification is able to stabilise convergence in case of oscillations
but sometimes the spectral shift is too small and then an increase in step size
would be appropriate, i.e.,
γ > 1. We propose a simple rule for adapting γ which
is inspired by predictive controllers used in robotics: a simple but slow and possi-
bly unstable reactive controller is used for teaching a new, predictive controller.
Usually stable and rapid convergence are difficult to achieve simultaneously, but
in this setup the new controller can be both faster and stabler.

Translated in our problem, the old slow and unstable controller is the weight
modification rule which proposes a modification of weight according to (10).
The new controller is implemented by (9), i.e., it modifies the step size. The
new controller tries to do immediately what the old controller would do in the
future. The step at the previous time instant was apparently optimal if the step
proposed at this time instant is orthogonal with it. If not,
γ should have been
different and, assuming that the optimal
γ is constant, the gamma used at this
time step should be

γnew = Yoid + wTdw∣∣woid∣∣2.                 (11)

As it does not seem productive to take steps in the direction opposite from what
is suggested by
w or to take extremely short steps, we require that γ 0.5.



More intriguing information

1. Ruptures in the probability scale. Calculation of ruptures’ values
2. Does adult education at upper secondary level influence annual wage earnings?
3. Types of Cost in Inductive Concept Learning
4. Linkages between research, scholarship and teaching in universities in China
5. A Rational Analysis of Alternating Search and Reflection Strategies in Problem Solving
6. The name is absent
7. The name is absent
8. The Role of Evidence in Establishing Trust in Repositories
9. The name is absent
10. NVESTIGATING LEXICAL ACQUISITION PATTERNS: CONTEXT AND COGNITION
11. How to do things without words: Infants, utterance-activity and distributed cognition.
12. Changing spatial planning systems and the role of the regional government level; Comparing the Netherlands, Flanders and England
13. The name is absent
14. The Challenge of Urban Regeneration in Deprived European Neighbourhoods - a Partnership Approach
15. Midwest prospects and the new economy
16. Equity Markets and Economic Development: What Do We Know
17. Do the Largest Firms Grow the Fastest? The Case of U.S. Dairies
18. The name is absent
19. A Note on Costly Sequential Search and Oligopoly Pricing (new title: Truly Costly Sequential Search and Oligopolistic Pricing,)
20. Determinants of U.S. Textile and Apparel Import Trade