financial services from another
programme.
Choosing your comparison group is
therefore very important and you must
do it carefully.
5 Design your
questionnaire
Once you have decided which issues the
impact survey is going to investigate,
you will need to decide what indicators
to use for measuring change. Indicators
are markers to help you to see if and
how changes are happening over a
period of time. They allow you to
establish benchmarks against which to
judge whether objectives are being met
(see Imp-Act Practice Note 5 on
Indicators).
Choosing indicators is not a simple
process. They must be relevant to your
programme’s needs and context, as well
as being sensitive to local conditions and
your MFI’s mission, and easy to identify
and monitor.
The hypotheses you have developed
and the objectives of the exercise should
guide you or the external consultant to
design a questionnaire which best meets
your needs, and which will provide you
with relevant, useful and timely results.
Most of the important decisions can be
made with help from your consultant,
but the following points will guide you in
good questionnaire design.
Keep it short: It should not take more
than 30 minutes to go through the
questionnaire with the respondent.
Keep it simple: Questions should be
simple and clearly phrased. Don’t
expect respondents to remember facts
from some time ago. Also make sure the
questions don’t result in complicated
answers that are difficult to analyse.
You need to think about how to translate
the questions into local languages and
make sure the meaning remains the
same.
Keep it relevant to your hypotheses
and maintain a focus on the key small
number of issues you want to explore.
Don’t forget that the main aim of the
survey is to produce impact data, rather
than client profile or satisfaction data.
Remember that the indicators you
choose may also be influenced by
findings from other impact assessment
tools you have used.
Do not collect data you can obtain
from elsewhere: Don’t ask for answers
that you could get from your
Management Information System (MIS)
- this will just make the questionnaire
longer.
Always aim to collect good quality
data: Questions should not be ‘leading’
- in other words, they should not be
looking for a particular answer.
Interviewers also need to think about
the time period or season to which
questions refer.
Think about how you will analyse
your data when designing the
questions: Are your questions
designed to produce answers that are
easy to analyse? Have you achieved the
right balance between closed and open-
ended questions to suit your needs? Is
there a standard way to record answers
which will help when comparing them?
(see Imp-Act Practice Note 2: QUIP:
Understanding clients through in-depth
interviews).
CASE STUDY 3 BENEFITS AND DRAWBACKS OF
USING IN-HOUSE STAFF
ODEF in Honduras has long-standing
experience in carrying out impact
surveys with in-house staff. ODEF
argues that the benefits of using their
credit officers for data collection, such
as cost-effectiveness and building on
existing knowledge, outweigh the
negative aspects such as the extra
work burden imposed on staff, the
possibility that staff might be biased
about their own clients, or that
respondents will motivated to give
untruthful answers. Together with the
Think carefully about the order of
your questions: You might wish to
leave sensitive issues, such as level
of satisfaction with the programme,
until last
Always Pilot test your questionnaire:
It is essential to test your questionnaire.
You need to know that correspondents
understand and can answer the
questions and that interviewers can
record answers easily. You also need
to make sure your staff can enter the
data into the computer and analyse it.
This will help you to fine-tune the
questionnaire to your needs, and focus
on the most relevant questions.
6 Plan your data
collection
The way in which you conduct interviews
and record responses will also affect the
reliability of the data. A number of basic
issues need to be considered about data
collection:
Who will carry out the data
collection? If this is to be your staff,
you will have to think about relieving
them from their usual workload burdens.
You may also want to give some thought
to the negative points involved in using
in-house staff (see Case study 3).
Covelo Network, ODEF is using its
experience to encourage other MFIs.
It emphasises that the negative
aspects can be countered in various
ways, including by keeping staff
informed about the processes, to
ensure their motivation, and by
careful training and close supervision.
IM P-ACT PRACTICE NOTES • NUMBER FOUR • 2005 • PAGE FIVE