put this another way, the node labels in a Connec-
tionist machine are not part of the causal structure
of the machine (Fodor and Pylyshyn, 1988, 13).
Only the activation levels of the input nodes and the
connection strengths in the network matter for an ANN
to produce the appropriate output for the function it
is trained to approximate. Nevertheless, labels for
the nodes and terminology for other parts of the net-
works are introduced whenever models are constructed.
Schultz (2003), for example, maps terms from neural nets
onto terms from developmental psychology (here, Piage-
tian theory).
Accommodation, in turn, can be mapped to
connection-weight adjustment, as it occurs, in the
output phase of cascade-correlation learning. [. . . ]
More substantial qualitative changes, corresponding
to reflective abstraction, occur as new hidden units
are recruited into the network. [. . . ] Then the net-
work reverts back to an output phase in which it
tries to incorporate the newly achieved representa-
tions of the recruited unit into a better overall solu-
tion. This, of course, could correspond to Piaget’s
notion of reflection (Schultz, 2003, 128, original ital-
ics).
The terminology from Piagetian theory clearly belongs
to a higher level of description than the true descriptions
of the network’s structure and dynamics.
Churchland (1998) suggests that a recurrent network
could model more challenging cognitive functions. He
considers that a recurrent network may have appropri-
ate architecture for simulating the acquisition of moral
virtues in humans. He argues that a network would be
able to map concepts like cheating, tormenting, lying, or
self-sacrifice within a n -space of classes containing di-
mensions of morally significant, morally bad, or morally
praiseworthy actions, by learning through “repeated ex-
posure to, or practice of, various examples of perceptual
or motor categories at issue” (Churchland, 1998, 83).
Churchland says that
[t]his high-dimensional similarity space [...] displays
a structured family of categorical “hot spots” or
“prototype position”, to which actual sensory in-
puts are assimilated with varying degree of closeness
(Churchland, 1998, 83).
It is beyond the scope of this paper to discuss whether
a model of such a calculus of moral virtues is appropri-
ate, but Churchland certainly demonstrates that it can
at least in principle be modeled with an ANN. However,
feed forward networks and SRNs with suitable numbers
of inputs and outputs and a reasonable number of hidden
units6 can be trained to implement almost any function.
The point here is that a network can implement almost
6Note that the number of hidden nodes and the number
of connections within the network are largely determined by
experience and experiment. There are no definite methods
or algorithms for this.
anything we want to model, as long as we have the ap-
propriate training set for the particular selection of la-
bels for the inputs and outputs of the ANN. As a result
we will have to accept that even simple ANNs provide
a universal, but uninformative, framework for cognitive
models.
There is a further methodological issue to consider. It
may be surprising to learn that neural nets in some mod-
els are not necessarily composed of neurons. Elman et al.
(1998) offer as a “note of caution” that
. . . [m]ost modelers who study higher-level cognitive
processes tend to view the nodes in their models as
equivalent not to single neurons but to larger popu-
lations of cells. The nodes in these models are func-
tional units rather than anatomical units (Elman
et al., 1998, 91).
Can models still be considered as ‘bottom-up’ neural
nets, if they are composed of functional units? I sug-
gest that such models do not belong in the realm of
connectionism, because the replacement of model neu-
rons with “functional units” re-introduces exactly those
black boxes that we trying to eliminate in the ‘bottom-
up’ approach.
The kinds of models that I am describing here, i.e. sim-
ple feed forward ANNs and small SRNs, do not rely
on special neural ‘circuitry’, unlike structured models in
which the models’ architectures reflect a particular part
of brain physiology. The architectures are universal in
the sense that only the number of neurons and connec-
tions vary from model to model. The diversity of models
that have been described in the literature is the product
of applying ANNs as analytical tools to a diverse set of
problems where suitable data sets for training of the net-
works are available. The universal architecture and the
freedom to choose labels and terminology fitting the par-
ticular model explains the proliferation of ANN inspired
models. Traditional mathematical (symbol based) mod-
els may be more constrained as far as the selection of
representations is concerned7. How then are explanatory
links maintained between representations in distributed
models and real world phenomena?
Symbols and Representations
Classic CMS are representational systems using sym-
bols, which carry arbitrarily assigned semantic content.
Haugeland (1985, 1981) and others have argued that
these semantics remain meaningful during processing, as
long as the syntactical structure is appropriate and suit-
ably maintained. Haugeland notes that in an interpreted
formal system with true axioms and truth-preserving
rules, the semantics will take care of itself, if you take
care of the syntax (Haugeland, 1981). The symbol ‘5’,
for example, carries different semantics in a positional
number system. Whether ‘5’ means ‘500’ in ‘1526’, or
‘50’ in ‘1257’ is a function that is governed by the syn-
tactic and semantic rules of the number system. Tying
7Dretske (1981, 1988), among others, has dealt with ques-
tions of representations and their semantics in representa-
tional systems.
1186