Understanding impact surveys
1 Decide what
information you want
Impact surveys can provide three types
of information:
1. Information about who your clients
are
2. The changes occurring in their lives
3. Their level of satisfaction with the
services offered
The main purpose of an impact survey is
to find out about changes in clients’ lives
and whether these are due to their
participation in the programme. There
are therefore a wide range of areas that
you may wish to investigate, for
example, changes in:
• Business type or size
• Household income and expenditure
• Asset ownership
• Housing and/or diet
• Expenditure on healthcare or education
• Behaviour or skills
• The ability to cope with unexpected
events or shocks which cause financial
distress
• Empowerment
• Community activities or increased
solidarity
Because there are so many areas that
you could examine, it is extremely
important to stay focused on the key
issues that your programme is
concerned about. These issues should
flow from the mission of your
programme: what were you hoping to
achieve in providing financial services to
your clients? For example, if one
programme aims to improve children’s
BOX 1 HYPOTHESES AND SURVEYS
A hypothesis is an expectation about
the effect that providing services will
have on your clients. For example, an
MFI might expect that “participation
in a microfinance programme leads to
increases in enterprise fixed assets” -
in which case a survey would aim to
test this hypothesis by including
nutrition as a result of giving women
credit and another focuses on expanding
businesses and business assets for men
and women, the questions you ask will
be different.
Before embarking on an impact
survey, you therefore need to decide:
What are the key issues about
changes in the lives of your clients
that you wish to focus on?
Be realistic. Don’t try to make up for a
lack of information by asking about
everything. Once you have identified the
key changes that you expect to have
occurred in clients’ lives and want to find
out about, these will form your
hypotheses (see box 1) from which to
build the survey.
How much of this information can
you get from other sources?
You may find that you don’t need to
include certain questions in your survey
because you already have the
information in your management
information system (MIS) or other
records.
Who is the report for? You will need
to decide whether the information is for
management or external audiences such
as donors, investors and the board,
because this will affect the issues you
focus on. You will also need to think
carefully about how you demonstrate
that the changes are due to your
programme’s activities, in other words
your approach to attribution. Attribution
means being able to make statements
about the causes of changes that you
observe in the lives of clients. Patterns
and directions of change may be useful
for management who want to use the
questions about fixed assets acquired
for the enterprise over the past year
and sources of funds used. Most MFIs’
mission statements have an MFI’s key
hypotheses embedded in them, but
these sometimes need to be made
more explicit.
findings to improve their operations.
People outside the organisation may
want a more statistical analysis that
shows the relationship between the
services provided and changes in clients’
lives.
Will you also conduct qualitative
research? Follow-up qualitative
research can provide more detailed
explanation of why changes are
happening. You need to think about the
approach that will best answer the
questions you have.
2 Decide what
resources you will
need
After deciding what you would like to
achieve, you will need to undertake in-
depth planning. Start by taking a
realistic look at the impact survey’s likely
requirements in terms of time, money
and skills.
Time: There are four main phases
involved in a research exercise such as
an impact survey: planning, design, data
collection and post-data collection (see
box 2). You will always need more time
than you think for data analysis and
report writing. Plan carefully to ensure
that you make the best use of this time.
Box 2 gives rough estimates of the time
involved for each phase.
Money: Surveys are a large investment
for most MFIs. Although they can be
reduced to focus on only a small number
of key issues to keep costs as low as
possible, you will need to carefully
calculate the costs involved in:
• Hiring external experts to help design
and supervise the survey and provide
training on data collection. Also for
analysis and report writing
• Computers and the software required
• Logistics for carrying out the survey -
vehicles, fuel, subsistence etc
• Covering the workload of staff
members involved in your research team
Skills: Impact surveys require a number
PAGE TWO • IMP-ACT PRACTICE NOTES • NUMBER FOUR • 2005