each other in some way so that it is possible, for example,
to look at the output of B - strings b - and infer something
about the behavior of Y from it - strings y. We suppose it
possible to define a retranslation from the B-language into
the Y-language through a deterministic code book, and call
Y the translated information source, as mirrored by B.
Define some distortion measure comparing paths y to paths
y, d(y, y) (Cover and Thomas, 1991). We invoke the Rate Dis-
tortion Theorem’s mutual information I (Y, Y), which is the
splitting criterion between high and low probability pairs of
paths. Impose, now, a parametization by an inverse coupling
strength K, and a renormalization symmetry representing the
global structure of the system coupling. This may be much
different from the renormalization behavior of the individual
components. If K < KC, where KC is a critical point (or
surface), the two information sources will be closely coupled
enough to be characterized as condensed.
In the absence of a distortion measure, we can invoke the
Joint Asymptotic Equipartition Theorem to obtain a similar
result.
We suggest in particular that detailed coupling mechanisms
will be sharply constrained through regularities of grammar
and syntax imposed by limit theorems associated with phase
transition.
Wallace and Wallace (1998, 1999) and Wallace (2002) use
this approach to address certain evolutionary processes in a
relatively unified fashion. These papers, and those of Wal-
lace and Fullilove (1999) and Wallace (2002a), further de-
scribe how biological or social systems might respond to gradi-
ents in information source uncertainty and related quantities
when the system is away from phase transition. Language-on-
network systems, as opposed to physical systems, appear to
diffuse away from concentrations of an ‘instability’ construct
which is related to a Legendre transform of information source
uncertainty, in much the same way entropy is the Legendre
transform of free energy density in a physical system.
Simple thermodynamics addresses physical systems held at
or near equilibrium conditions. Treatment of nonequilibrium,
for example highly dynamic, systems requires significant ex-
tension of thermodynamic theory. The most direct approach
has been the first-order phenomenological theory of Onsager,
which involves relating first order rate changes in system pa-
rameters Kj to gradients in physical entropy S, involving ‘On-
sager relation’ equations of the form
∑ Rk,j dKj/dt = ∂S∕∂Kj,
k
where the Rk,j are characteristic constants of a particular
system and S is defined to be the Legendre transform free
energy density F;
S ≡ F - ∑ ∂F∕∂Kj.
j
The entropy-analog for an information system is, then, the
dimensionless quantity
S ≡ H - ∑ Kj ∂H∕∂Kj,
j
or a similar equation in the mutual information I .
Note that in this treatment I or H play the role of free
energy, not entropy, and that their Legendre transform plays
the role of physical entropy. This is a key matter.
For information systems, a parametized ‘instability’,
Q[K] ≡ S - H, is defined from the principal splitting cri-
terion by the relations
Q[K] = -KdH[K]/dK
Q[K] = -KdI[K]/dK
(4)
where H[K] and I[K] are, respectively, information
source uncertainty or mutual information in the Asymptotic
Equipartition, Rate Distortion, or Joint Asymptotic Equipar-
tition Theorems.
Extension of thermodynamic theory to information systems
involves a first order system phenomenological equations anal-
ogous to the Onsager relations, but possibly having very com-
plicated behavior in the Rj,k, in particular not necessarily
producing simple diffusion toward peaks in S . For example,
as discussed, there is evidence that social network structures
are affected by diffusion away from concentrations in the S-
analog. Thus the phenomenological relations affecting the
dynamics of information networks, which are inherently open
systems, may not be governed simply by mechanistic diffusion
toward ‘peaks in entropy’, but may, in first order, display more
complicated behavior.
2. ‘Biological’ phase transitions. Now the mathemati-
cal detail concealed by the invocation of the asymptotic limit
theorems emerges with a vengeance. Equation (2) states that
the information source and the correlation length, the degree
of coherence on the underlying network, scale under renor-
malization clustering in chunks of size R as
H[KR, JR]/f(R) = H[J, K]
χ[KR, JR]R = χ(K, J),
with f(1) = 1, K1 = K, J1 = J, where we have slightly
rearranged terms.
Differentiating these two equations with respect to R, so
that the right hand sides are zero, and solving for dKR/dR
and dJR/dR gives, after some consolidation, expressions of
the form
dKR/dR = u1d log(f)/dR + u2/R
dJR∕dR = vι JRd log(f )∕dR + R2 Jr.
(5)