The Interaction of Modes
In addition to mapping the semiotic resources of the modes and looking at how these
resources realise meaning (meta-functions) multimodal analysis requires attending to
how modes interact in a communicative event. One approach to this, that I mentioned
earlier is to examine how the resources of modes work together to realise ideational,
interpersonal, and textual meaning, that is three meta-functions. Mapping how the
semiotic resources of each mode in a communicative event or text contribute to these
specific kinds of meanings is my starting point in the thesis.
The discrete but interconnected concepts of functional specialisation of modes and
functional load offer a tool for analysing how these modal resources contribute to
meaning of a Communicational event or text. I employ the concept ‘functional
specialisation’ to theorise the specialised representational and Communicational
functions of modes at a general level. The concept ‘functional load’ is used to analyse
the distribution of information across the modes in the specific instances of learning
that I analyse in the thesis. The concept of modes being ‘foregrounded’ or
‘backgrounded’ in a communicative event or text provides a further tool for the
analysis of the interaction of modes. Using these concepts I analyse the interaction of
modes including the use of different modes to reinforce the ‘same’ message, to carry
or elaborate different aspects of the overall message, or to convey contradictory
aspects of a message.
The semiotic principles of discourse, design, production, and dissemination that
Kress and van Leeuwen (2001) argue stand behind multimodal communication and
operate across modes are used as a framework for thinking about meaning making in
the multimodal context of computer mediated learning. The concept of design is
most useful for the purposes of this thesis. I use the concept of ‘design’ to explore
how discourses are formulated in computer applications through the combining of
specific semiotic modes in particular ways, to explore the realisation of discourses of
the designers of the computer application and the texts made by the students.
80