we will impose phase transition characterized by a renormal-
ization symmetry, in the sense of Wilson (1971).
We will call such an information source ‘adiabatically piece-
wise memoryless ergodic’ (APME).
To anticipate the argument, iterating the analysis on paths
of ‘tuned’ sets of renormalization parameters gives a second
order punctuation in the rate at which primary interacting in-
formation sources representing cognitive submodules become
linked to each other: the shifting workspace structure of con-
sciousness.
Interacting cognitive modules
We suppose that a two (relatively) distinct cognitive sub-
modules can be represented by two distinct sequences of
states, the paths x ≡ x0, x1, ... and y ≡ y0, y1, .... These
paths are, however, both very highly structured and seri-
ally correlated and have dual information sources X and Y.
Since the modules, in reality, interact through some kind
of endless back-and-forth mutual crosstalk, these sequences
of states are not independent, but are jointly serially cor-
related. We can, then, define a path of sequential pairs as
z ≡ (x0, y0), (x1, y1), The essential content of the Joint
Asymptotic Equipartition Theorem (JAEPT), a variant of the
Shannon-McMillan Theorem, is that the set of joint paths z
can be partitioned into a relatively small set of high probabil-
ity termed jointly typical, and a much larger set of vanishingly
small probability. Further, according to the JAEPT, the split-
ting criterion between high and low probability sets of pairs
is the mutual information
I(X,Y) = H(X) -H(X|Y) = H(X) +H(Y) -H(X,Y)
where H(X), H(Y ), H(X|Y ) and H(X, Y ) are, respec-
tively, the (cross-sectional) Shannon uncertainties of X and
Y , their conditional uncertainty, and their joint uncertainty.
See Cover and Thomas (1991) for mathematical details. Sim-
ilar approaches to neural process have been recently adopted
by Dimitrov and Miller (2001).
Note that, using this asymptotic limit theorem approach,
we need not model the exact form or dynamics of the crosstalk
feedback, hence crushing algebraic complexities can be post-
poned until a later stage of the argument. They will, however,
appear in due course with some vengeance.
The high probability pairs of paths are, in this formulation,
all equiprobable, and if N (n) is the number of jointly typical
pairs of length n, then
I(X,Y) = lim ''(n)].
n→∞ n
Extending the earlier language-on-a-network models of
Wallace and Wallace (1998, 1999), we suppose there is a cou-
pling parameter P representing the degree of linkage between
the modules, and set K = 1/P , following the development
of those earlier studies. Note that in a brain model this pa-
rameter represents the intensity of coupling between distant
neural structures.
Then we have
I[K] =
lim log[N(K, n)]
n→∞ n
The essential ‘homology’ between information theory and
statistical mechanics lies in the similarity of this expression
with the infinite volume limit of the free energy density. If
Z(K) is the statistical mechanics partition function derived
from the system’s Hamiltonian, then the free energy density
is determined by the relation
F[K] = lim log[Z(K)].
V→∞ V
F is the free energy density, V the system volume and K =
1/T , where T is the system temperature.
We and others argue at some length (Wallace and Wallace,
1998, 1999; Rojdestvensky and Cottam, 2000) that this is
indeed a systematic mathematical homology which, we con-
tend, permits importation of renormalization symmetry into
information theory. Imposition of invariance under renormal-
ization on the mutual information splitting criterion I(X, Y)
implies the existence of phase transitions analogous to learn-
ing plateaus or punctuated evolutionary equilibria. An exten-
sive mathematical development will be presented in the next
section.
The physiological details of mechanism, we speculate, will
be particularly captured by the definitions of coupling param-
eter, renormalization symmetry, and, perhaps, the distribu-
tion of the renormalization across agency, a matter we treat
below.
Here, however, these changes are perhaps better described
as ‘punctuated interpenetration’ between interacting cogni-
tive modules.
We reiterate that the details are highly dependent on the
choice of renormalization symmetry and distribution, which
are likely to reflect details of mechanism - the manner in
which the dynamics of the forest are dependent on the detailed
physiology of trees, albeit in a many-to-one manner. Renor-
malization properties are not likely to follow simple physical
analogs, and may well be subject, in addition to complications
of distribution, to the ‘tuning’ of universality class parameters
that are characteristically fixed for simple physical systems.
The algebra is straightforward if complicated, and given later.
Representations of the general argument
1. Language-on-a-network models. Earlier papers of
this series addressed the problem of how a language, in a large
sense, spoken on a network structure responds as properties of
the network change. The language might be speech, pattern
recognition, or cognition. The network might be social, chem-
ical, or neural. The properties of interest were the magnitude
of ‘strong’ or ‘weak’ ties which, respectively, either disjointly
partitioned the network or linked it across such partitioning.
These would be analogous to local and mean-field couplings
in physical systems.
We fix the magnitude of strong ties - to reiterate, those
which disjointly partition the underlying network (presum-
ably into cognitive submodules) - but vary the index of weak
ties between components, which we call P , taking K = 1/P .
For interacting neural networks P might simply be taken as
proportional to the degree of crosstalk.
We assume the piecewise, adiabatically memoryless ergodic
information source (Wallace, 2002b) depends on three param-
eters, two explicit and one implicit. The explicit are K as