26
Joachim Funke
2, I will briefly describe the philosophy underlying this research, the DY-
NAMIS shell for creating and presenting scenarios, the general experimental
procedure, the diagnostic approach to eliciting subjects’ knowledge about the
task, and the approach to measuring performance. In Section 3 the results of
five studies within this framework will be presented. Finally, in Section 4 the
results will be summarized and related to other studies. Also, I will give some
perspectives for future research.
2. A Method for Analyzing Complex Problem Solving
From its beginning, research about solving complex problems had to cope
with a number of difficulties (see the critical aspects mentioned by Funke,
1984). One central point was the reliable measurement of problem solving
quality. Since no optimal solution path and no “best” intervention was avail-
able in most of these microworlds (because of the partially nonlinear rela-
tionships between the variables for which mathematically no optimal solution
could be found), researchers were never quite sure whether a subject’s solu-
tion to a problem was really better or worse than that of other subjects.
However, at least qualitative judgments were possible. In other cases, in
which subjects could set their own goals, problem solving quality was rated
by “experts.” Along with this came a complete loss of comparability of
results. With these tasks it was impossible to separate out which part of the
observed system changes was due to the tasks’ characteristics and which was
due to the subjects’ attempt to cope with the problem. Also, the question of
reliability of performance measures has been answered mainly by referring
to the face validity of the tasks.
To overcome some of these problems, the line of research done in our
Bonn laboratory established the following principles: (1) It should always be
possible to define the quality of a solution by comparing it with an optimal
solution strategy. (2) The situation should realize the features of complex
problems (complexity, connectivity, intransparency, Eigendynamik [i.e., au-
tonomous processes] and multiple goals) as far as possible. (3) A detailed
diagnostic procedure should reveal subject’s development of hypotheses
about the system. This implies that subjects have to be prompted repeatedly
about the causal structure they assume to the system. (4) There should be a
clear distinction between a phase of knowledge acquisition (mainly realized