6. Conclusions
In this paper, we developed a low level value sys-
tem for a developmental robot. Both simulation and
real-time experiments are reported. The value sys-
tem integrates the habituation mechanism and re-
inforcement learning. We successfully applied the
system to simulate visual attention effect. Our SAIL
robot learns to pay attention to salient visual stim-
uli while neglecting unimportant input. Motivated
by psychology studies in instrumental conditioning,
we integrated reinforcement learning with habitua-
tion so that the robot’s responses to certain visual
stimuli would change after interacting with human
trainers, that is, cognitive development of the robot
takes place. Even though the low level value system
modeled some adaptive behaviors in animal learning,
what we accomplished is still one step towards the
challenging autonomous mental development. Our
next step is to implement the SHM (Stagger Hierar-
chical Mapping) (Zhang et al., 2001) method to do
local analysis and apply the framework to vision-
based outdoor navigation.
Acknowledgments
The authors would like to thank Wey S. Hwang for
his major contribution to an earlier version of the
IHDR program and Yilu Zhang for his contribution
to the Q-Iearning algorithm. The work is supported
in part by National Science Foundation under grant
No. IIS 9815191, DARPA ETO under contract No.
DAAN02-98-C-4025, and DARPA ITO under grant
No. DABT63-99-l-0014.
References
Almassy, N., Edelman, G., and Sporns, O. (1998).
Behavioral constraints in the development of
neural properties: A cortical model embedded in
a real-world device. Cerebral Cortex, 8(4):346-
361.
Domjan, M. (1998). The Principles of learning
and behavior. Brooks/Cole Publishing Com-
pany, Belmont, CA.
Flavell, J., Miller, P., and Miller, S. (1993). Cog-
nitive Development. Prentice-Hall, Englewood
Cliffs, NJ.
Hwang, W. and Weng, J. (1999). Hierarchical dis-
criminat regression. IEEE Trans, on Patten
Analysis and Machine Intelligence, 22(11): 1277—
1293.
Kaplan, P., Werner, J., and Rudy, J. (1990).
Habitutation, sensitization and infant visual at-
tention. In Rovee-Collier, C. and Lipsit, L.,
(Eds.), Advances in Infancy Research, pages 61-
110. ABLEX Publishing Corporation, Norwood,
NJ.
Montague, P., Dayan, P., and Sejnowski, T. (1996).
A framework for mesencephalic dopamine sys-
tems based on predictive hebbian learning. The
journal of Neuroscience, 16(5):1936-1947.
0gmen, H. (1997). A developmental perspective to
neural models of intelligence and learning. In
Levine, D. and Elsberry, R., (Eds.), Optimal-
ity in Biological and Artificial Networks ?, pages
363-395. Lawrence Erlbaum Associates, Pub-
lishers, Hillsdale, NJ.
Piaget, J. (1952). The Origins of Intelligence in
Children. International Universities Press, INC,
New York.
Rucci, M., Tononi, G., and Edelman, G. (1997).
Registration of neural maps through value-
dependent learning: Modeling the alighment of
auditory and visual maps in the barn owl’s pitic
tecturm. Journal of Neuroscience, 17:334-352.
Schultz, W. (2000). Multiple reward signals in the
brain. Nature Reviews: Neuroscience, 1:199—
207.
Sporns, O. (2000). Modeling development and
learning in autonomous devices. In Workshop
on Development and Learning, pages 88-94, E.
Lansing, Michigan, USA.
Sporns, O., Almassy, N., and Edelman, G. (2000).
Plasticity in value system and its role in adap-
tive behavior. Adaptive Behavior.
Sur, M., Angelucci, A., and Sharm, J. (1999).
Rewiring cortex: The role of patterned activ-
ity in development and plasticity of neocortical
circuits. Journal of Neurobiology, 41:33-43.
Sutton, R. S. and Barto, A. (1998). Reinforcement
Learning - An Introduction. The MIT Press,
Chambridge, MA.
Watkins, C. (1992). Q-Iearning. Machine Learning,
8:279-292.
Weng, J. and Hwang, W. (2000). An incremen-
tal learning algorithm with automatically de-
rived discriminating features. In Proc, of Fourth
Asian Conference on Computer Vision, pages
426-431, Taipei, Taiwan.
Weng, J., McClelland, J., Pentland, A., Sporns, O.,
StockMan, I., Sur, M., and Thelen, E. (2000).
Autonomous mental development by robots and
animals. Science, 291:599-600.