I(X|Y)=H(X)+H(Y)-H(X,Y),
which contains products of terms and their logs, sub ject
to constraints that the sums of probabilities are 1 and each
probability is itself between 0 and 1. Maximization is done
by varying the channel matrix terms P (yj |xi) within the con-
straints. This is a difficult problem in nonlinear optimization.
However, for the special case M = L, C * may be found by
inspection:
If M = L, then choose
P (yj |xi) = δj,i ,
where δi,j is 1 if i = j and 0 otherwise. For this special case
C* ≡ H(X),
with P (yk) = P(xk) for all k. Information is thus trans-
mitted without error when the channel becomes ‘typical’ with
respect to the fixed message distribution P (X).
If M < L, matters reduce to this case, but for L < M infor-
mation must be lost, leading to Rate Distortion limitations.
Thus modifying the channel may be a far more efficient
means of ensuring transmission of an important message than
encoding that message in a ‘natural’ language which maxi-
mizes the rate of transmission of information on a fixed chan-
nel.
We have examined the two limits in which either the dis-
tributions of P (Y ) or of P (X) are kept fixed. The first pro-
vides the usual Shannon Coding Theorem, and the second a
tuning theorem variant, a tunable retina-like Rate Distortion
Manifold. It seems likely, however, than for many important
systems P(X) and P(Y ) will interpenetrate, to use Richard
Levins’ terminology. That is, P (X) and P (Y ) will affect each
other in characteristic ways, so that some form of mutual tun-
ing may be the most effective strategy.
19 Acknowledgments
The author thanks Dr. C. Guerrero-Bosagna for comments
useful in revision.
20 References
Ash R., 1990, Information Theory, Dover Publications, New
York.
Atlan, H., and I. Cohen, 1998, Immune information,
self-organization, and meaning, International Immunology,
10:711-717.
Atmanspacher, H., 2006, Toward an information theoreti-
cal implementation of contextual conditions for consciousness,
Acta Biotheoretica, 54:157-160.
Baars, B.,1988, A Cognitive Theory of Consciousness,
Cambridge University Press, New York.
Baars, B., 2005, Global workspace theory of conscious-
ness: toward a cognitive neuroscience of human experience,
Progress in Brain Research, 150:45-53.
Backdahl, L., A. Bushell, and S. Beck, 2009, In-
flammatory signalling as mediator of epigenetic modu-
lation in tissue-specific chronic inflammation, The In-
ternational Journal of Biochemistry and Cell Biology,
doi:10.1016/j.biocel.2008.08.023.
Bennett, M., and P. Hacker, 2003, Philosophical Founda-
tions of Neuroscience, Blackwell Publishing.
Bennett, C., 1988, Logical depth and physical complexity.
In The Universal Turing Machine: A Half-Century Survey,
R. Herkin (ed.), pp. 227-257, Oxford University Press.
Bos, R., 2007, Continuous representations of groupoids,
arXiv:math/0612639.
Bossdorf, O., C. Richards, and M. Pigliucci, 2008, Epige-
netics for ecologists, Ecology Letters, 11:106-115.
Britten, R., and E. Davidson, 1969, Gene regulation for
higher cells: a theory, Science, 165:349-357.
Brown, R., 1987, From groups to groupoids: a brief survey,
Bulletin of the London Mathematical Society, 19:113-134.
Buneci, M., 2003, Representare de Groupoizi, Editura Mir-
ton, Timisoara.
Cannas Da Silva, A., and A. Weinstein, 1999, Geometric
Models for Noncommutative Algebras, American Mathemati-
cal Society, RI.
Champagnat, N., R. Fierriere, and S. Melard, 2006, Uni-
fying evolutionary dynamics: from individual stochastic pro-
cesses to macroscopic models, Theoretical Population Biology,
69:297-321.
Ciliberti, S., O. Martin, and A. Wagner, 2007a, Robust-
ness can evolve gradually in complex regulatory networks with
varying topology, PLoS Computational Biology, 3(2):e15.
Cohen, I., 2006, Immune system computation and the im-
munological homunculus. In Nierstrasz, O., J. Whittle, D.
Harel, and G. Reggio (eds.), MoDELS 2006, LNCS, vol. 4199,
pp. 499-512, Springer, Heidelberg.
Cohen, I., and D. Harel, 2007, Explaining a complex living
system: dynamics, multi-scaling, and emergence. Journal of
the Royal Society: Interface, 4:175-182.
Cover, T., and J. Thomas, 1991, Elements of Information
Theory, John Wiley and Sons, New York.
Crews, D., and J. McLachlan, 2006, Epigenetics, evolu-
tion, endocrine disruption, health, and disease, Endocrinol-
ogy, 147:S4-S10.
Crews, D., A. Gore, T. Hsu, N. Dangleben, M. Spinetta,
T. Schallert, M. Anway, and M. Skinner, 2007, Transgenera-
tional epigenetic imprints on mate preference, Proceedings of
the National Academy of Sciences, 104:5942-5946.
Dehaene, S., and L. Naccache, 2001, Towards a cognitive
neuroscience of consciousness: basic evidence and a workspace
framework, Cognition, 79:1-37.
Dembo, A., and O. Zeitouni, 1998, Large Deviations: Tech-
niques and Applications, 2nd edition, Springer, New York.
Dias, A., and I. Stewart, 2004, Symmetry groupoids and
admissible vector fields for coupled cell networks, Journal of
the London Mathematical Society, 69:707-736.
24