A Multimodal Framework for Computer Mediated Learning: The Reshaping of Curriculum Knowledge and Learning

Ideational Meaning

Movement and gesture can be classified into different kinds of ‘narrative processes’
in a similar way to visual communication. The ‘processes in action’ can be described
as either ‘doing’ which is realised by some kind of movement, or ‘state’ which is
realised by a lack of movement a stillness (Martinec, 2000: 314). When an action is
realised it can either be directed at another person or object (e.g. the screen or mouse)
or поп-directed (without a goal). As in the mode of image, the question of who or
what is acting and who or what they are acting on is central to the semiotic analysis of
movement and gesture or classroom interaction more broadly speaking.

Alongside the analysis of the processes of action (movement and gesture) is what
Martinec calls ‘aspects of the processes’ which he defines as speed and force
(Martinec, 2000: 315). Drawing on the materiality of movement and gesture, I expand
the notion of ‘aspect’ in this thesis to include the dimensions of ‘openness’ versus
‘closeness’, ‘roughness’ versus ‘smoothness’, ‘tense’ versus ‘lax’, and ‘direction’. At
a first and most general level these dimensions have associated semiotic values. The
speed of a gesture can be fast or slow and the degree of deliberation expressed, can
either be done to signify the competence (habituated action) or the confidence of the
person making it or their indifference. If a movement or gesture is rough and jerky it
may signify a lack of control or the difficulty of a task. In contrast a smooth gesture
may be used to signify control and ease. In addition to narrative processes, like
images, movement and gesture can realise conceptual meanings. Movements and
gestures can be ‘symbolic’, for example the wave of the Toontalk robots as a gestural
appeal to the user for action (see Chapter Five).

Interpersonal Meaning

The variation of ‘distance’ and body angle between people (and/or the objects that
they interact with) realise different kinds of interpersonal relations or what Martinec
calls ‘degrees of engagement’ (Martinec, 2000: 317). The semiotic resources of social
distance and ‘attitude’ (angle) described earlier in relation to visual and aural

77

<<   74   75   76   77   78   79   80   >>

More intriguing information