40
Joachim Funke
bility seems possible, then at least replications should be presented in order
to establish stable phenomena.
Traditional measurement theory is not the only way to give answers to this
question. Modem concepts like latent state-trait theory (see Steyer, Widaman,
& Graser, in press), with their distinction between consistency and specificity
of certain variables, offer the potential for careful design of reliability studies
such as the one conducted by Miiller (in press) which revealed good scores
for the QSC and QSI measures but also showed the existence of situational
influences.
One possible line of validity research could be the use of the learning test
concept (e.g., Guthke, 1982) according to which intelligence is not a static
variable but must be interpreted as “learning potential.” It could be possible
that relations exist between this “learning potential” and the ability to solve
complex problems. Following a solid comparison of the two different re-
quirements, predictions are possible concerning what kind of selected de-
pendent variables from both areas should correlate.
Concerning task 3, the adequate measuring of the problem solvers’ mental
models and their heuristics, one has to develop instruments that sensitively
assess those relevant parts of human memory that are required for identifica-
tion and control. Whereas in the area of assessing structural (or declarative)
knowledge some useful techniques exist, there are clear deficits in diagnosing
the heuristic knowledge that human problem solvers operate with. Also, more
attention should be given to developing measurement techniques that reveal
the implicit knowledge of an operator. These could be developed similar to
the procedures used in experimental memory research: If the recent use of a
representation or of a procedure facilitates its subsequent access (as revealed,
for instance, by reaction time measurements in verification tasks on these
representations or procedures), then indirect effects could be established in
roughly the same manner as in research on human memory.
Reaction time measurements have proven useful in the area of process
control of finite state automata (see Funke & Buchner, in press). Based on a
formal analysis of the task under study it was demonstrated that a facilitating
effect on RT occurs in verification situations in which the “natural” sequence
of state transitions is reinstated during test.
Concerning the general research strategy, I find it more useful to manipu-
late critical variables in system structures and in presentation modes than to
create numerous new systems which are completely unrelated and offer no
solid basis for comparisons. Collecting data without theoretical assumptions
produces puzzling situations in which spurious correlations may suggest
significant effects where no effects are present. Only the strategy of analyzing
the effects of selected variations based on some minimal theoretical prem-
ises—the experimental method—can offer new insights into the principles
and mechanisms that govern complex human problem solving. For this pur-
pose, the research strategy outlined above offers a method for the systematic