Connectionism, Analogicity and Mental Content
certain high-level physical properties of real neural networks. Most obviously, the variable
activation levels of processing units and the modifiable weights on connections in PDP networks
directly reflect the spiking frequencies of neurons and the modulatory effects of synaptic
connections, respectively.8 Terrence Sejnowski goes further, arguing that while PDP systems do
not attempt to capture molecular and cellular detail, they are nonetheless “stripped-down
versions of real neural networks similar to models in physics such as models of ferromagnetism
that replace iron with a lattice of spins interacting with their nearest neighbors” (1986, p.388). As
with any idealisation in science, what goes into such an account depends on what properties of
neural nets one is trying to capture. The idealisation must be complex enough to do justice to
these properties, and yet simple enough that these properties are sufficiently salient (see, eg,
Churchland & Sejnowski 1992, chp.3). In this respect, the PDP framework isolates and hence
enables us to focus on the computationally significant properties of neural nets, while ignoring
their fine-grained neurochemistry. Our best neuroscience informs us that neural nets compute
by generating patterns of neural activity in response to inputs, and that these patterns of activity
are the result of the modulatory effects of synapses in the short-term, and modifications to these
synapses over the longer term. It’s precisely these structural and temporal properties that are
captured by the networks of processing units and connection weights that comprise PDP
systems.
In spite of this strong hint, however, most theorists have been reluctant to press the
concept of analogicity into service in their descriptions of how connectionism differs from
classicism. This is due primarily, I think, to the fact that the PDP computational framework
resists easy classification on the standard account of the digital/analog distinction that one
encounters in the literature: since the material substrate of a PDP model comprises connection
weights and unit activation values that need not take on continuous values, it’s often not
obvious whether this substrate realises a continuous or discrete representational medium. But
this all changes when we move to the alternative conception of the digital/analog distinction
that I’ve outlined. Viewed from this perspective, the issue is not whether these models employ
continuous or discrete representational media, but how they produce semantically coherent
behaviour. And the PDP models discussed in the connectionist literature do this by first
constructing and then exploiting structural isomorphisms between their substrates and their
representational domains, not by performing rule-governed manipulations of symbolic
representations. It is clear, in other words, that PDP networks, at least as they are standardly
employed, 9 are analog computational devices.10
8 See, eg, Rumelhart and McClelland 1986, chp.4. There are other connectionists who are not entirely happy with this
interpretation, however. Paul Smolensky, for example, argues that because we are still largely ignorant about the
dynamical properties of the brain that drive cognitive operations, and because the PDP framework leaves out a
number of properties of the cerebral cortex, connectionism is situated at a level once removed from real neural
networks (1988).
9 This qualification is necessary, of course, because it is well known that a PDP network can implement a Turing
machine. When employed in this fashion, PDP networks operate as digital computers.
10 The analog character of the PDP framework is also obscured by the fact that most PDP networks are simulated on
digital computers. In such simulations, the activation values that compose a network’s activation pattern together with
the values of the connection weights are typically recorded in complex arrays, each of whose elements is subject to
updating according to the algorithms that model the network’s activity. But these data structures are not equivalent to
a pattern of activation or a the configuration of connection weights across a real (non-simulated) PDP network. The
latter are structures constructed from physically connected elements (such as neurons), each of which realises a
continuously variable physical property (such as a spiking frequency) of a certain magnitude. The former, by contrast,
are symbolic representations of such structures, in that they consist of a set of discrete symbol structures that “describe”
in a numerical form the individual activation levels of a network’s constituent processing units together with the
values of its connection weights. The activation landscape embodied in a real network thus has a range of complex
structural properties (and consequent causal powers) that are not reproduced by the data structures employed in
simulations. This fact is most vividly demonstrated by the temporal asymmetries that exist between real PDP