Towards Learning Affective Body Gesture

Table 1: The table summarizes subjects’ observations,
necessary, (-): irrelevant, (∖): not desired.

Amplitude (A), Rigidity (R). The complexity feature
refers to the number of body parts that were involved
in the motion. It was observed that, in some cases,
the simplification of the body movement was making
the recognition of the affective state more difficult.
The speed feature was pointed out as important in
the case of angry and sad animations. In the first
case, a slow tapping of the foot was perceived as
a rhythmical accompanying movement. Symmetry
and irregularity were shown to be very important
features both for the naturalness of the gesture and
for effectively conveying an affective state. In these
cases, the movement was repeated continuously with-
out any variation. Rigidity not only resulted in the
motion being effectively seen as unnatural but also
decreased the intelligibleness of the affective state.

3. Capturing Emotional States

We implemented a system that incrementally learns
the mapping function between body postures and
emotional labels. A video camera captures a human
made expressive posture and sends it to the compu-
tational system. The system analyzes and creates
a posture signature that describes the relative posi-
tion of the body joints. Details about the posture
description are in (Bianchi-Berthouze et al., 2003).
However, the algorithm proposed is very simple, and
is not the focus of our work at this stage. The signa-
ture is hence mapped into an emotional label. The
recognized emotion can be further used, according
to the motivation or goal of the system, to select an
appropriate reaction, e.g. to play a specific piece of
music to change the user’s state.

We see the mapping of posture features into
emotional labels as a categorization prob-
lem. We propose to use a modified version
(Berthouze and Tijsseling, 2002) of Categorizing
and Learning modules (CALM) (Murre et al., 1992),
that can self-organize inputs into categories. A
CALM network consists of several CALM modules.
While the topology of a CALM architecture is
fixed, connections between modules are learnt. To
improve its performance, the system can receive
two types of feedback from its user. The first type
of feedback is verbal feedback and is sent by the
user directly to the system to explicitly indicate the
correct emotional label when it outputs the wrong
label. The second type of feedback is postural
feedback and corresponds to changes in the user
body postures as a reaction to the system’s actions.

4. Conclusions

We tested the Mood Mapping module with 108 dif-
ferent postural images of an agent with marked joints
involving 3 types of affective postures: happy, angry
and sad postures. Only 1 error occurred: the posture
was related to a sad emotion but was classified as an-
gry by the model. The final aim of this work would
see the system as an active actor in the interaction
with the human. To increase the complexity of the
possible postures to be modeled, we are now begin-
ning to exploit motion features by analyzing data
acquired with both a 3D motion capture system and
video. This will require us to deal with the temporal
dimension of the motion features already detected as
relevant in our preliminary experiments.

References

Berthouze, L. and Tijsseling, A. (2002). Acquir-
ing ontological categories through interaction.
SDForum, 16(4):141-147.

Bianchi-Berthouze, N., Fushimi, T., Hasegawa, M.,
Kleinsmith, A., Takenata, H., and Berthouze,
L. (2003). Learning to recognize affective body
postures. Proc, of the IEEE Intl. Symposium
on Computational Intelligence for Measurement
Systems and Applications, Lugano, Switzerland.

Bianchi-Berthouze, N. and Lisetti, C. (2002). Mod-
eling multimodal expression of users’ affective
subjective experience. International Journal on
User Modeling and User-Adapted Interaction,
12(1):49-84.

Breazeal, C. and Scassellati, B. (1999). How to
build robots that make friends and influence
people. International Conference on Intelligent
Robots and Systems,Kyonjiu, Korea.

Murre, J., Phaf, R., and Wolters, G. (1992). Calm:
Categorizing and learning module. Neural Net-
works, 5:55-82.

Nakata, T. (2002). Generation of whole-body ex-
pressive movement based on somatical theories.
Proceedings of the 2nd international workshop
on epigenetic robotics, pages 105-114.

Picard, R. (1997). Affective Computing. MIT Press.

von Laban, R. (1988). The mastery of movement.

Princeton.

<<   <   1   2