aspects affect the European economy at a more global level.
Some generalizations of neural network models `a la Tsallis were already pre-
viously reported (Cannas, Stariolo, & Tamarit, 1996; Hadzibeganovic & Can-
nas, 2007; submitted). These generalizations are based on analogies between
the properties of neural network models and those found in statistical physics
and thermodynamics. As discussed by Hopfield (1982) and then applied to at-
tractor networks by Amit and colleagues (1985), neural network models have
direct analogies in statistical physics, where the investigated system consists
of a large number of units each contributing individually to the overall, global
dynamic behavior of the system. The characteristics of individual units rep-
resent the microscopic quantities that are usually not directly accessible to
the observer. However, there are macroscopic quantities, defined by parame-
ters that are fixed from the outside, such as the temperature T = 1∕β and
the mean value of the total energy. The main aim of statistical physics is to
provide a link between the microscopic and the macroscopic levels of an inves-
tigated system. An important development in this direction was Boltzmann’s
finding that the probability of occurrence for a given state {x} depends on
the energy E ({x}) of this state through the well-known Boltzmann-Gibbs dis-
tribution P({x}) = Zexp[-βE({x})], where Z is the normalization constant
Z = ∑{x} exp[-βE({x})].
In the context of neural networks, statistical physics can be applied to study
learning behavior in the sense of a stochastic dynamical process of synaptic
modification (Watkin, Rau, & Biehl, 1993). In this case, the dynamical vari-
ables {x} represent synaptic couplings, while the error made by the network
(with respect to the learning task for a given set of values of {x} ) plays the
role of the energy E({x}). The usage of a gradient descent dynamics as a
synaptic modification procedure leads then to a stationary Boltzmann-Gibbs
distribution for the synapses (Watkin et al., 1993). However, the gradient de-
scent dynamics corresponds to a strictly local learning procedure, while non
local learning dynamics may lead to a synaptic couplings distribution different
from the Boltzmann-Gibbs one (Stariolo, 1994; Cannas, Stariolo, & Tamarit,
1996).
Here, we briefly report an implementation of the Tsallis entropy1 formalism
in a simple neural network model which has been used for simulation of learning
behavior in adults. In this model, a generalization of the gradient descent
dynamics is realized via a nonextensive cost function (Stariolo, 1994) defined
by the map
V = q ln[1 + β(q - i)v ] (4)
where the index q is an arbitrary real number such that q ≥ 1; V is a monoton-
ically increasing function of V, and therefore it preserves its minima structure.
The Langevin equation, which governs the (local) gradient descent dynamics
that is usually applied in neural networks, is here replaced by:
dJij _ 1 ∂V
(5)
~dΓ = — 1+ β(q - 1)V ∂J~j + ηij (t)
1 Tsallis’ entropy Sq =
ι-∑ i pq
q-
(q ∈ R) is a nonlogarithmic (generalized) entropy, which
reduces to standard Boltzmann-Gibbs-Shannon entropy SBGS = - i pi ln pi as the nonex-
tensive entropic index q approaches unity. See Appendix C for a proof and details.