lim H(Xo,...,Xn)
n→∞ n
We say this information source is dual to the ergodic cognitive process.
Different ‘languages’ will, of course, be defined by different divisions of
the total universe of possible responses into different pairs of sets B0 and
B1 , or by requiring more than one response in B1 along a path. Like the
use of different distortion measures in the Rate Distortion Theorem (e.g.
[14]), however, it seems obvious that the underlying dynamics will all be
qualitatively similar.
Here, meaningful paths - creating an inherent grammar and syntax - are
defined entirely in terms of system response, as Atlan and Cohen [3] propose.
See [56] for explicit application of this formalism to mathematical models of
neural process.
We will eventually parametize the information source uncertainty of this
dual information source with respect to one or more variates. We can write
H[K], where K ≡ (K1 , ..., Ks ) represents a vector in an appropriate param-
eter space. Let the vector K follow some path in time, i.e. trace out a
generalized line or surface K(t). We will, following the argument of Wal-
lace (2002b), assume that the probabilities defining H, for the most part,
closely track changes in K(t), so that along a particular ‘piece’ of a path in
parameter space the information source remains as close to memoryless and
ergodic as is needed for the mathematics to work. Between pieces we impose
phase transition characterized by a renormalization symmetry, in the sense
of Wilson [60].
We will call such an information source ‘piecewise memoryless ergodic’.
Iterating the argument on paths of ‘tuned’ sets of renormalization param-
eters gives a second order punctuation in the rate at which primary inter-
acting information sources come to match each other in a distorted manner,
the essence of adaptation or interpenetration.
2. Information dynamic phase transitions The essential homology
relating information theory to statistical mechanics and nonlinear dynamics
has been described elsewhere [42, 51, 52, 57], and we truncate the discussion
here, although Feynman [20] shows in great detail that, for certain simple
physical systems, the homology is, in fact, an identity.
The definition of the free energy density for a parametized physical system
is
10