Connectionism, Analogicity and Mental Content
10
Consider, as a simple example, the mine/rock detecting network discussed by Paul
Churchland (1988, pp.157-62). The network takes as input a “sonar echo” emanating from either
a rock or a mine (sampled across 13 different frequencies and thus coded across 13 input units),
processes this through a hidden layer (of 7 hidden units), and produces an output (across 2
output units) which should indicate the source of the echo. The network is initially exposed to a
range of rock echoes (which are superficially different) and a range of mine echoes (likewise),
and is trained up using the back propagation learning procedure. This procedure so modifies the
network’s connectivity pattern, that it soon becomes capable of distinguishing between the two
general types of echoes. But how does it do this? Numerical analysis of the activation patterns
generated over the hidden units of the trained up network reveals that they cluster in two
disjoint regions of activation space, and depending on which region is activated the network
outputs either “rock” or “mine”. The back propagation learning procedure has thus moulded the
network’s material substrate, through subtle modifications to its pattern of connectivity, that it
now contains an activation landscape that systematically mirrors abstract properties of the
“sonar echoscape” (for want of a better term). In other words, the trained up network embodies
an analog of its representational domain, in that its representational vehicles (the connection
weight representations stored in its pattern of connectivity, which manifest themselves in the
form of network activation patterns) are structurally isomorphic (in a second order sense) with
objects in this target domain (the “abstract” properties in common to mine and rock echoes,
respectively).
Or consider, as another example, NETtalk, probably the most talked about PDP model in
the connectionist literature (Sejnowski and Rosenberg, 1987). NETtalk transforms English
graphemes into their appropriate phonemes, given the context of the words in which they
appear. The task domain, in this case, is even more abstract, in that it is the letter-to-sound
correspondences that exist in the English language. But NETtalk’s operating principles are
similar to those found in the rock/mine detector. Again, back propagation is used to shape its
activation landscape, this time consisting of patterns across 80 hidden units, such that,
eventually, a structural isomorphism obtains between its representational substrate and the
target domain. It is this isomorphism that is revealed in the now very familiar cluster analysis to
which Sejnowski and Rosenberg subjected NETtalk. It is this isomorphism that makes it right
and proper to talk, as everyone does, of a semantic metric across NETtalk’s activation landscape.
Furthermore, it is this isomorphism that provides NETtalk with its computational power: when
the light of a grapheme, embedded in an array of graphemes, is shone on NETtalk’s surface, it
automatically casts the appropriately contextualised phonemic pattern. This PDP network is
thus an analog computational device in just the same way as the scale model we examined
earlier: its material substrate embodies a complex “scale model” of the letter-to-sound
correspondences found in English.
Nothing could be clearer than that these networks do not transform symbols according to
syntactically applied rules; than that they are not digital computers. Yet they process their
representational vehicles in a semantically coherent fashion. They are able to do this because
their material substrates have been moulded by learning procedures into “shapes” that resemble
aspects of their representational domains. When such resemblance relations are in place, their is
no need to force these networks to conform to rules which dictate how their representational
vehicles are to be processed. Instead, semantic coherence emerges quite naturally in these
networks, driven by the causal laws that apply to the materials from which they are constructed.
networks and their digital simulations: the simulations are notoriously slow at processing information, when
compared to their real counterparts, in spite of the incredible computational speed of the digital machines on which
they are run. The bottom line here is that a simulated network does not embody an analog of its representational
domain, and hence is no more an analog device than a simulated hurricane is a hurricane.