A Multimodal Framework for Computer Mediated Learning: The Reshaping of Curriculum Knowledge and Learning

We move away from the idea that the different modes in multimodal
texts have strictly bounded and framed specialist tasks, as in a film
where images may provide the action, sync sounds a sense of realism,
music a layer of emotion, and so on, with the editing process
supplying the ‘integration code’, the means for synchronising the
elements through a common rhythm (van Leeuwen, 1985). Instead we
move towards a view of multimodality in which common semiotic
principles operate in and across different modes, and in which it is
therefore quite possible for music to encode action, or images to
encode emotion.

(Kress and van Leeuwen, 2001: 2)

Kress and van Leeuwen (2001) put forward four interconnected semiotic principles
for consideration in multimodal analysis including discourse, design, production, and
distribution. I draw on these as analytical tools throughout the thesis.

Discourse is used to refer to ‘socially situated forms of knowledge about (aspects of)
reality’ (Kress and van Leeuwen, 2001: 20). Different discourses are available to
people in relating a particular event and they use the discourse that is most apt to
express their interests in the Communicational situation that they are in. These
discourses construct knowledge through the inclusion and exclusion of participants
and events, and the links that are made with other events and discourses. Discourses
can only be realised in semiotic modes and this introduces the concept of design.

Design is a process that stands between content and expression and is used to refer to
‘conceptualisations of the form of semiotic products and events’ (Kress and van
Leeuwen, 2001: 21). Design involves the formulation of discourse in a particular
social context through the combining of semiotic modes in a particular way. In short,
design is the means for the realisation of discourse and is separate from the actual
material product.

Kress and van Leeuwen use the concept of Production to consider ‘the articulation in
material form of semiotic products or events’ (Kress and van Leeuwen, 2001: 21). It

44

<<   41   42   43   44   45   46   47   >>

More intriguing information