2. Turning a Multimodal Lens on Technology-Mediated
Learning
Introduction
In order to understand the relationship between the multimodal semiotic resources
that new technologies make available in the classroom, and knowledge, literacy,
design and learning, I turn to social semiotic theory as it theorises the relationship
between semiotic resources and people’s meaning making. Halliday’s (1978) social
semiotic theory focuses primarily on the semiotic resources of language (speech and
writing). However, I argue that the prominent role of visual semiotic resources on the
computer screen (and elsewhere) and the multimodal character of learning requires a
theory of meaning making that moves beyond language. For this reason I take a
multimodal approach to the analysis of technology-mediated learning which builds on
the theoretical concepts of social semiotics in order to account for the semiotic
resources of a range of modes including still image, colour, animated movement,
writing, sound-effect, and speech.
While social semiotic and multimodal theory offer conceptual tools for the analysis of
meaning making, the theoretical focus is on ‘semiosis’. As such the socially situated
nature of meaning making in schools is not foregrounded. The rules and norms that
underpin meaning making in schools and the traditions and histories that are
embedded in curriculum and subject knowledge are not clearly in their analytical
domain. In order to give due attention to the socially situated character of meaning
making I theorise multimodal semiotic mediation within the analytical framework of
activity theory. In order to focus on specific instances of ‘formal’ learning I focus on
school, pedagogy and curriculum.
37