The minimization is over all conditional distributions p(y∣y)
for which the joint distribution p(y,y) = p(y)p(y∣y) satisfies
the average distortion constraint (i.e. average distortion ≤
D).
The Rate Distortion Theorem states that R(D) is the max-
imum achievable rate of information transmission which does
not exceed the distortion D. Cover and Thomas (1991) or
Dembo and Zeitouni (1998) provide details.
More to the point, however, is the following: Pairs of se-
quences (yr,yr) can be defined as distortion typical; that is,
for a given average distortion D , defined in terms of a partic-
ular measure, pairs of sequences can be divided into two sets,
a high probability one containing a relatively small number
of (matched) pairs with d(yn,yrn) ≤ D, and a low probabil-
ity one containing most pairs. As n → ∞, the smaller set
approaches unit probability, and, for those pairs,
p(yr) ≥ p(yr∣yr)eχp[-nl(Y,Y)].
(28)
Thus, roughly speaking, I (Y, Y) embodies the splitting cri-
terion between high and low probability pairs of paths.
For the theory of interacting information sources, then,
I(Y, Y) can play the role of H in the dynamic treatment that
follows.
The rate distortion function can actually be calculated in
many cases by using a Lagrange multiplier method - see Sec-
tion 13.7 of Cover and Thomas (1991).
References
Aiello W., F. Chung, and L. Lu, 2000, A random graph
model for massive graphs, in Proceedings of the 32nd Annual
ACM Symposium on the Theory of Computing.
Albert R., and A. Barabasi, 2002, Statistical mechanics of
complex networks, Reviews of Modern Physics, 74:47-97.
Asanovic K, R. Bokik, B. Catanzaro, J. Gebis, P.
Husbands, K. Keutzer, D. Patterson, W. Plishker, J.
Shalf, S. Williams, and K. Yellick, 2006, The landscape
of parallel computing research: a view from Berkeley,
http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-
2006-183.pdf.
Ash R., 1990, Information Theory, Dover Publications,
New York.
Atlan H., and I. Cohen, 1998, Immune information ,self-
organization and meaning, International Immunology, 10:711-
717.
Auslander L., 1967, Differential Geometry, Harper and
Row, New York.
Baars B., 1988, A Cognitive Theory of Consciousness,
Cambridge University Press, New York.
Baars B., and S. Franklin, 2003, How conscious experience
and working memory interact, Trends in Cognitive Science,
doi:10.1016/S1364-6613(03)00056-1.
Baars, B., 2005, Global workspace theory of conscious-
ness: toward a cognitive neuroscience of human experience,
Progress in Brain Research, 150:45-53.
Bak A., R. Brown, G. Minian and T. Porter, 2006, Global
actions, groupoid atlases and related topics, Journal of Ho-
motopy and Related Structures, 1:1-54. Available from ArXiv
depository.
Bennett M., and P. Hacker, 2003 Philosophical Foundations
of Neuroscience, Blackwell Publishing, London.
Brown R., 1987, From groups to groupoids: a brief survey,
Bulletin of the London Mathematical Society, 19:113-134.
Burago D., Y. Burago, and S. Ivanov, 2001, A Course
in Metric Geometry, American Mathematical Society, Provi-
dence, RI.
Cohen I., 2000, Tending Adam’s Garden: Evolving the Cog-
nitive Immune Self, Academic Press, New York.
Cohen T., B. Blatter, C. Almeida, E. Shortliffe, and V. Pa-
tel, 2006, A cognitive blueprint of collaboration in context:
Distributed cognition in the psychiatric emergency depart-
ment, Artificial Intelligence in Medicine, 37:73-83.
Connes A., 1994, Noncommutative Geometry, Academic
Press, San Diego.
Corless R., G. Gonnet, D. Hare, D. Jeffrey, and D. Knuth,
1996, On the Lambert W function, Advances in Computa-
tional Mathematics, 4:329-359.
Cover T., and J. Thomas, 1991, Elements of Information
Theory, John Wiley and Sons, New York.
Dehaene S., and L. Naccache, 2001, Towards a cognitive
neuroscience of consciousness: basic evidence and a workspace
framework, Cognition, 79:1-37.
Dehaene S., and J. Changeux, 2005, Ongoing spontaneous
activity controls access to consciousness: a neuronal model
for inattentional blindness, PLOS Biology, 3:e141.
Dembo A., and O. Zeitouni, 1998, Large Deviations: Tech-
niques and Applications, Second edition, Springer, New York.
Dretske F., 1981, Knowledge and the Flow of Information,
MIT Press, Cambridge, MA.
Dretske F., 1988, Explaining Behavior, MIT Press, Cam-
bridge, MA.
Dretske, F., 1993, Mental events as structuring causes of
behavior, in Mental Causation (ed. by A. Mele and J. Heil),
pp. 121-136, Oxford University Press.
Dretske F., 1994, The explanatory role of information,
Philosophical Transactions of the Royal Society A, 349:59-70.
Erdos P., and A. Renyi, 1960, On the evolution of random
graphs, reprinted in The Art of Counting, 1973, 574-618 and
in Selected Papers of Alfred Renyi, 1976, 482-525.
Feynman, R., 1996, Feynman Lectures on Computation,
Addison-Wesley, Reading, MA.
Freeman, W., 2003, The wave packet: an action potential
of the 21st Century, Journal of Integrative Neurosciences, 2:3-
30.
Fullilove, M., 2004, Root Shock, Ballantine Books, New
York.
Glazebrook, J., 2006, Rate distortion manifolds as model
spaces for cognitive information. In preparation.
16