realisation that little more can be learnt and that the study is over, or that the study
needs radical revision and iteration to an earlier phase(s), or progression to a
subsequent phase. The overall programme might be envisaged as tending towards an
artefact or ‘product’ of some kind. This product might be a theory (if the desired
outcome is simply knowledge), a proposed improvement for public policy, or a
tool/resource for a practitioner. In order for any of these outcomes to be promoted
and disseminated in an ethical manner they must have been tested (or else the
dissemination must merely state that they seem a good idea but that we have no real
idea of their value). A theory, by definition, will generate testable propositions. A
proposed public policy intervention can be tested realistically and then monitored in
situ for the predicted benefits, and for any unwanted and undesirable side effects.
Therefore, for that minority of programmes which continue as far as phase 6 in
Figure 1, rigorous testing must usually involve a mixture of methods and types of
evidence in just the same way as phase 1. Even where a purely numeric outcome is
envisaged as the benefit of the research programme (such as a more effective or cost-
efficient service) it is no good knowing that the intervention works if we do not also
know that it is unpopular and likely to be ignored or subverted in practice. Similarly,
it would be a waste of resource, and therefore unethical, simply to discover that an
intervention did not work in phase 6 and so return to a new programme of study in
phase 1. We would want to know why it did not work, or perhaps how to improve it,
and whether it was effective for some regular pattern of cases but not for others. So in
phase 6, like phase 1, the researcher or team who genuinely wants to find something
out will naturally use a range of methods and approaches including measurement,
narrative and observation.
The same kind of conclusion could be reached for every phase in Figure 1. Even
monitoring and evaluation of the rollout of the results (phase 7) is best done by using
all and any data available. Even in simple academic impact terms, a citation count for
a piece of research gives no idea of the way in which it is used (just mentioned or
fundamental to the new work of others), nor indeed whether the citation is critical of
the research and whether it is justified in being critical. On the other hand, for a
widely cited piece of research, reading in-depth how the research has been cited in a
few pieces gives no idea of the overall pattern. Analysing citation patterns and
reading some of the citing pieces - perhaps chosen to represent features of the overall