tive process near and away from ‘critical points’ in the cou-
pling of cognitive submodules. The question of whether we
are demonstrating the necessity of global phase transitions in
information-transmission networks or merely building a sug-
gestive analogy with thermodynamics is an empirical one be-
yond our present ability to answer. For the microscopic case,
however, Feynman (1996) has shown that the homology is an
identity, which is no small matter and indeed suggests that
behavior analogous to phase transitions in simple physical sys-
tems should be ubiquitous for certain classes of information
systems.
Our work is likely analogous, in a certain sense, to Bohr’s
treatment of the atom, which attempted a simple substitu-
tion of quantized angular momentum into a basically classi-
cal theory. Although incomplete, that analysis contributed
materially to the more comprehensive approaches of quan-
tum mechanics, relativistic quantum mechanics, and quantum
electrodynamics. In that spirit we hope that increasingly sat-
isfactory models will follow from the interplay of our work
here and appropriate empirical studies.
We begin with a description of cognitive process in terms
of an information source, a kind of language constrained by
the Shannon-McMillan or Asymptotic Equipartition Theo-
rem, and its Rate Distortion or Joint Asymptotic Equipar-
tition and other variants for interacting sources.
Cognition as language
Atlan and Cohen (1998) and Cohen (2000), following a long
tradition in the study of immune cognition (e.g., Grossman,
1989; Tauber, 1998), argue that the essence of cognitive func-
tion involves comparison of a perceived signal with an inter-
nal, learned picture of the world, and then, upon that compar-
ison, the choice of a response from a much larger repertoire of
possible responses. Following the approach of Wallace (2000,
2002a), we make a ‘weak’, and hence very general, model of
that process.
Cognitive pattern recognition-and-response, as we charac-
terize it, proceeds by convoluting an incoming external sen-
sory incoming signal with an internal ongoing activity - the
learned picture of the world - and triggering an appropriate
action based on a decision that the pattern of sensory activ-
ity requires a response. We will, fulfilling Atlan and Cohen’s
(1998) criterion of meaning-from-response, define a language’s
contextual meaning entirely in terms of system output, leav-
ing out, for the moment, the question of how such a pattern
recognition system is trained, a matter for Rate Distortion
theory.
The abstract model will be illustrated by two neural net-
work examples.
A pattern of sensory input is mixed in some unspecified but
systematic manner with internal ‘ongoing’ activity to create
a path of convoluted signal x = (a0, a1, ..., an, ...). This path
is fed into a highly nonlinear, but otherwise similarly unspec-
ified, decision oscillator which generates an output h(x) that
is an element of one of two (presumably) disjoint sets B0 and
B1 of possible system responses. We take
B0 ≡ b0,...,bk,
B1 ≡ bk+1,..., bm.
Thus we permit a graded response, supposing that if
h(x) ∈ B0
the pattern is not recognized, and if
h(x) ∈ B1
the pattern is recognized and some action bj , k + 1 ≤ j ≤ m
takes place.
We are interested in paths x which trigger pattern
recognition-and-response exactly once. That is, given a fixed
initial state a0 , such that h(a0) ∈ B0 , we examine all possi-
ble subsequent paths x beginning with a0 and leading exactly
once to the event h(x) ∈ B1 . Thus h(a0, ..., aj) ∈ B0 for all
j < m, but h(a0, ..., am) ∈ B1.
For each positive integer n let N (n) be the number of
paths of length n which begin with some particular a0 having
h(a0) ∈ B0 and lead to the condition h(x) ∈ B1 . We shall
call such paths ‘meaningful’ and assume N (n) to be consid-
erably less than the number of all possible paths of length n
- pattern recognition-and-response is comparatively rare. We
further assume that the finite limit
H ≡ ,im log[N(n)l
n→∞ n
both exists and is independent of the path x. We will -
not surprisingly - call such a pattern recognition-and-response
cognitive process ergodic. Not all such processes are likely to
be ergodic, implying that H, if it exists, is path dependent,
although extension to ‘nearly’ ergodic processes is straight-
forward.
Invoking Shannon, we may thus define an ergodic in-
formation source X associated with stochastic variates Xj
having joint and conditional probabilities P(a0, ..., an) and
P(an|a0, ..., an-1) such that appropriate joint and conditional
Shannon uncertainties may be defined which satisfy the rela-
tions
H[X] = lim log[N(n)l =
n→∞ n
lim H(Xn|X0, ..., Xn-1) =
n→∞
lim H(Xo,...,Xn)
n→∞ n
(1)
The Shannon uncertainties H (...) are defined in terms of
cross-sectional sums of the form - k Pk log[Pkl, where the
Pk constitute a probability distribution. See Ash (1990) or
Cover and Thomas (1991) for details.
We say this information source is dual to the ergodic cog-
nitive process.