image of a novel, playing cards, and so on serve to visually signify the subject
English as a world of social knowledge, history and tradition. In the multimodal
computer application, Multimedia Science School meaning is realised ‘between’ the
mode of image, the limited written and numerical elements, and the movement of the
elements. The use of visual resources in the application suggests the need to re-think
(re-visualise) the world as it is usually experienced. The visual representation in
science, of the world beyond the usual experience of people, stands in contrast to the
visual association with the past in the CD-ROM designed for use in school English.
Through the use of colour, image, writing, and action in the applications designed for
different school subjects different discourses are produced and realised around
knowledge and the subjectivity of the learner. The illustrative examples of computer
applications that are discussed in this thesis demonstrate the need for a multimodal
theory that accounts for the complexity of the relations between modes.
Just as there are different conventions of how image and writing are displayed on
particular pages (the pages of school books and newspapers, and so on) some
patterned configurations of modal resources on screen are beginning to emerge.
Although these modal conventions change over time (like all conventions) the
relationship between image and writing on a newspaper page for instance is markedly
different in 2003 than it was in say 1960 (Kress, 2003). At the time of writing this
thesis, however, the conventions of modal arrangements on screen remain unstable
and emergent conventions.
The Multimodal Engagement of Students with New Technology
This thesis demonstrates the benefits of looking beyond students’ talk around the
computer. In addition to the multimodal character of the majority of computer
applications (including Microsoft Word that this thesis is being written on, with its
animated office assistant and visual icons, etc.) students engage with these
applications multimodally. They point, gesture, gaze at the screen, move the mouse
286