326
J. Funke
Discussion of Experimental Results
The three experiments described above have something in common: they all
demonstrate the differential effects of subtle changes in system attributes and
task requirements on the dependent variables:
(1) Active intervention leads to a better control performance, but
Verbalizableknowledgedecreases (Exp. 1).
(2) To let subjects predict the next system state increases theiramount
of verbalizable knowledge (Exp. 1).
(3) Knowledge predicts performance (Exp. 1, 3).
(4) Growing Eigendynamik deteriorates control performance but not
the available knowledge (Exp. 2).
(5) Growing side effects reduce control performance and available
knowledge (Exp. 3).
Thus, comparing side effects with Eigendynamik as two important variables
contributing to the complexity of systems it shows up that Eigendynamik can
be more easily detected, but Less easily controlled. In the case of side effects
the situation is more complicated: subjects are not able to detect the cause of
changes in the system variables correctly and, thus, have little chance of good
control. As Miiller (1993) points out, these problems in the identification of
system structure lead to a phenomenon called “compensatory assumptions’’.
According to this hypothesis, subjects try to build up a model which explains
the available system data by means of incorrect models; the incorrectness
comes from simple, but wrong causal paths which compensate for the right,
but complicated ones. In his data analysis, MUller (1993) could demonstrate
that subjects either have the right model about the implemented side effects
(which seldom occurs) or have a certain simple wrong model which explains
the system changes with compensatory assumptions.
Even if subjects would not detect different degrees of side effects or
Eigendynamik - an argument used recently by Strohschneider (1991) in a
critique of the presented experimental approach - the effects of this
manipulation on the dependent measures cannot be denied. Also, the result of
Brehmer (1989; Brehmer & Allard, 1991) point to the critical role of system
characteristics. In their studies, introducing different degrees of feedback
delay leads to a detrimental problem solving behavior (see also Funke, 1985;
Stermanl 1989).
Chapter 14 Micrmvoriiis Based on Linear Equation Systems
327
Discussion of DYNAMiS approach
Besides the results of the above reported experiments, there are some general
features of the Bonn approach to complex problem solving worth discussing.
The main progress made in the three years of the DYNAMIS project can be
summarized as follows:
(I) It was possible to develop and run an experimentally oriented
research methodology in the area of complex problem solving.
This guarantees causal interpretation of the reported effects.
(2) A formal framework exists for the description and construction of
arbitrary dynamic systems with continuous variables. This
guarantees comparability between studies with different systems
(as done in Exp. 1, 2, and 3).
(3) Measures for quality of system control and quality of system
identification have been derived which show acceptable
psychometric quality. This guarantees -in combination with
acceptable power of statistical tests- that the non-appearance of
certain effects cannot be attributed to unreliable measures.
(4) The effects of subtle changes in system attributes and task
demands have been demonstrated experimentally with respect to
their consequences for identification and control.
(5) A first step towards a taxonomy of influence factors in dealing
with dynamic systems has been made as an attempt to integrate
the results (for more details see Funke, 1990).
Until now, there has not been impressive theoretical progress in this research
domain. But the instruments for doing research and a corresponding
framework have been developed which seem to be the basis for theoretical
work. As often occurs in the development of science, the preparation of
analytical tools was a necessary step for further insights (for example, even
the traditional theory of finite state automata could be a useful and new tool
for problem solving research, see Buchner & Funke11993; Funke & Buchner,
1992). This step has been done.
Concerning the general research strategy, it seems more useful to manipulate
critical variables in system structures and in presentation modes than to create
numerous of new systems which are completely unrelated and offer no solid
basis for comparisons. Also, replications of reported effects are quite
necessary - this requirement also applies to the experiments reported here.