A Multimodal Framework for Computer Mediated Learning: The Reshaping of Curriculum Knowledge and Learning

The Semiotic Resources of Sound

Like colour the resources of sound are less clearly specialised along meta-functional
lines than language and visual communication, however the modes of voice, sound-
effect and music do realise the three meta-functions and is treated as a mode. In
western cultures, however, sound cannot be treated in the same way as language and
visual communication as it has not reached the '...levels of abstraction and functional
saturation that (written) language and image have reached...' (van Leeuwen, 1999:
192). Theoretically the meta-functions apply in the same way but to a less fully
articulated mode; and therefore the range of meaning potential is less constrained.
Van Leeuwen uses the material aspects of sound rather than its communicative
functions as an entry point to identify the semiotic resources of sound. The modes of
voice and sound effect are two modes that feature in the analysis that I offer in this
thesis. The main semiotic resources that I draw on from van Leeuwen’s work are
briefly outlined below.

The semiotic resource of ‘social distance’ described in some detail earlier in relation
to visual semiotics can be used to think about sound. The social distance between the
listener and (the maker of) a sound is realised by its volume, aural nearness and sound
quality. The semiotic quality of an intimate whisper, a far off shout or a public
announcement each signifies a different kind of social relation between the maker of
the sound and its receiver. Sounds also mark interaction through the use of what van
Leeuwen calls ‘sonic interaction’ (van Leeuwen, 1999). This means that sound can
be either monologic or dialogic (between individuals, groups, etc.).

Sound time is a semiotic resource that organises aural texts. A sound can be either a
fluctuating or continuous unmeasured sound or a rhythmic measured regular sound
(with many variations of what this can mean). Another semiotic resource of sound
used in this thesis is ‘sound quality’, that is whether a sound is tense or lax, loud or
soft, rough or smooth, and so on.

75

<<   72   73   74   75   76   77   78   >>

More intriguing information