A Multimodal Framework for Computer Mediated Learning: The Reshaping of Curriculum Knowledge and Learning

(orjoystick) and click on icons and, sometimes, they talk. Just as the modal elements
on screen offer different affordances for the construction of curriculum knowledge so
do the different modes of interaction with the computer screen.

When a student makes a gestured ‘mark’ across the screen to represent their imagined
movement of an element, for instance, this gesture is temporary and ephemeral. It
does not demand the modal commitment (the permanence) of a drawn or animated
line of trajectory with the resources of an application. Throughout the instances of
technology-mediated learning analysed in this thesis students used gesture to plan and
imagine, and to test theories and ideas. Through gesture the students ‘created’ a space
of activity that overlaid the computer screen. ‘Within’ such applications the spatial
resources of the screen itself become meaningful and these were used to indicate
different kinds of activity, such as constructing elements, note taking and so on. In
the case of the Steinbeck CD-ROM and Multimedia Science School these areas were
designed and designated by the makers of the application, in the case of Toontalk
these areas were created by the interaction of the students. Once the students engaged
with the mouse or keyboard the character of their gestures were transformed to more
‘permanent’ realisations of plans and instructional gestures that, were they to be
‘translated’ into words, would be along the lines of ‘go here’ ‘click here’, or ‘select
this’.

Gaze is a resource employed by students in the organisation of their interaction with
the screen often visualised in the movement of the cursor on screen. Gaze is a
resource that software designers have used to engage the user since the early tool bars
of the Mac Classic with ‘eyes’ that ‘follow’ the cursor on the screen, through to the
humanoid robots of Toontalk. These material signs realised through gaze can, I argue,
be interpreted as a multimodal sign of interest, attention and intention both of the
designer and the students. I have shown that by attending to the mode of gaze what
may at first be taken as lack of engagement with a text, for example, flicking quickly
through the screens of a CD-ROM, can be understood as a kind of engagement.

287

<<   285   286   287   288   289   290   291   >>

More intriguing information