similar ongoing emergence). We include bootstrapping as
a criterion for ongoing emergence because it seems
inevitable that a robot either needs to have some means of
spontaneously developing its own set of initial skills or,
more conventionally, will need to have some initial skills
pre-programmed prior to its being turned on. While pre-
programming of bootstrap skills is not consistent with the
concept of skills being developed by the agent itself, we
consider this an acceptable practice if for no other reason
than keeping the scope of research projects tractable.
However, in our view, the preferred method to establish
bootstrapping skills is to represent those skills in the
same manner as later emerging skills such that both the
bootstrapping and developed skills comprise a uniform
part of the agent’s skill repertoire.
Stability of skills is in part a practical matter: in order
for a skill to be measured (i.e., by researchers), it must
exist for a measurable duration. In terms of the robot,
however, stability may be more than merely a practical
matter in that, in order for ongoing emergence to be
achieved, a certain degree of skill stability over time will
likely prove necessary. If the behaviors exhibited by the
robot are merely transient then those behaviors may not
contribute to the basis for the acquisition of new skills.
The reproducibility criterion asks the question: Under
what starting-state and environmental conditions does a
given developmental algorithm produce an ongoing
emergence of behavior? We presume that if a
developmental algorithm is well-understood, then the
starting-state and conditions under which it produces
ongoing emergence will also be well-understood. These
conditions may be limited (e.g., to specific values of
initial variables), but once known can reproducibly
generate ongoing emergence.
3. Current Research & Ongoing
Emergence
In this section, we review examples of epigenetic robotic
research (see Table 2) in light of our criteria for ongoing
emergence. Our selection of these particular papers is not
intended to reflect some a prior sense that they have
achieved ongoing emergence. Rather they simply reflect
our subjective impression of good illustrative examples
of research in this area.
Several lines of research satisfy Criterion 1 (new skill
acquisition). For example, a swinging behavior emerges
in the robot of Berthouze and Lungarella (2004), and the
skill of tracking a face view to an object emerges in the
robot of Nagai et al. (2003). To varied extents, some
research has also satisfied Criterions 3 through 6.
Criterion 3 (autonomy of goals and values) is satisfied to
some degree by the robot of Seth et al. (2004), and also
the work of Kaplan and Oudeyer (2003). Seth et al.
(2004) utilize a value system in the Darwin VIII robot to
signal the occurrence of salient sensory events. Initially,
Darwin VIII’s value system was activated by sounds
detected by the robot’s auditory system, but through
learning became activated by particular visual stimulus
attributes. Criterion 4 (bootstrapping) is satisfied by the
Dominguez and Jacobs (2003) system in that the system
uses progressive changes in visual acuity to increase its
binocular disparity sensitivity. Criterion 5 (stability)
appears satisfied, for example, by the Lungarella and
Berthouze (2002) system in that the robots’ swinging
behavior reaches stable states. Criterion 6
(reproducibility) is satisfied by studies which replicated
their robots’ behavior, perhaps under varied conditions.
For example, Chen and Weng’s (2004) experiments were
replicated with 12 separate robot “subjects” (the same
robot and algorithms, but with different environmental
conditions).
To give a more extended example of how these
criteria can be applied, we consider the work of Nagai et
al. (2003). These authors modeled joint visual attention
behavior using a robot. Joint attention occurs when
individuals both look at the same object, and may include
knowledge of shared attentional states (e.g., Carpenter,
Nagell, & Tomasello, 1998). The Nagai et al. (2003)
robot learned to track the face view of a person to the
object the person was looking at. Learning started with
the robot (a) knowing how to visually detect faces and
salient objects, (b) knowing to switch its gaze from a face
to an object when both the object and face were in its
field of view, and (c) having a predefined transition
function (a sigmoidal) to switch from how salient objects
were found—either directly in its visual field, or
indirectly by first looking at a face3. Initially the robot did
not know how much to turn its head based on a particular
view of a face to find the object that the person was
looking at, and learning acquired this skill. The transition
function enabled this skill to be gradually applied.
The Nagai et al. (2003) robot has some behaviors that
are programmed into the system (e.g., bootstrapping,
Criterion 4). A new behavior is constructed from the
initial behaviors (e.g., visually detecting faces and
objects) and environmental interaction—i.e., the robot
learns to track faces to the objects that they are looking at
(Criterion 1 is satisfied). However, once the new behavior
has emerged, there is no further potential for
development. That is, the new skill was not incorporated
into the system in such a way that it contributed to the
basis for further skill development. Thus, Criterion 2 is
not satisfied. Reproducibility (Criterion 6) was
demonstrated in the system by conducting experimental
runs with 1, 3, 5, or 10 objects in which the emergent
behavior was maintained. In summary, while some of the
criteria for ongoing emergence are satisfied, the behavior
of the Nagai et al. (2003) robot seems best classified as
demonstrating emergence and not ongoing emergence.
Notably absent in this review of current work is full
evidence for Criterion 2 (incorporation of new skills with
existing skills so that those new skills can be used as part
of the basis for further development). We have yet to find
examples of robots exhibiting this ability (but we hope to
be corrected on this point!). This leads us to view current
3 The use of this sigmoidal seems unnecessary, but in our view should
not be viewed as a shortcoming of this research. In principle, the
authors could have used a method of self-supervised learning to
transition between modes: when the robot was sufficiently able to
predict the amount of head turn required for accurately turning to face
an object, it could have then begun utilizing its self-generated head turn.
65