Using Surveys Effectively: What are Impact Surveys?



Designing a survey methodology

4 Assess changes in
clients’ status

The objective of an impact survey is to
be able to make statements about how
far changes in the lives of clients are
due to participation in a programme,
rather than to external factors. To
achieve this, your survey must allow
comparisons to be made either:
Between those who use your MFI’s
services and those who do not (a
comparison group - similar to a
control
group in experimental sciences). This is
known as a
cross-sectional study
and/or

Between the circumstances of clients
before they joined the programme
(provided by baseline data) compared to
their circumstances after they have used
the services for a period of time. This is
known as a
longitudinal study.

Longitudinal studies usually make
comparisons over time - and have the
advantage of looking at changes in the
lives of the
same clients. However, they

BOX 3: THE AIMS/SEEP IMPACT SURVEY TOOL


The AIMS/SEEP impact survey tool is
an example of a “middle-range”
survey tool, designed to look at the
types of issues MFIs may want to
investigate about impact, but at a
more modest cost than many
surveys. Its broad features are the
following:

It is implemented at one point in
time rather than repeated over time.
It asks the same questions of
clients who have been in the
programme for different periods of
time.

It is used with clients who have just
entered the programme but not yet
received a loan as the main
comparison group.

It collects information on four
aspects of clients’ lives: about them
as individuals; their households; their
require baseline information (i.e. from
before the client joined the programme),
which can then be compared with
information collected at a later date to
see what the changes are. This usually
takes a considerable amount of time
since starting to collect baseline
information now means that findings will
only be available after the clients are
again interviewed again at a future date
- usually one or two years.

Cross-sectional studies usually make
comparisons between different groups.
They are usually preferred by MFIs
because they can provide findings in a
shorter time frame. These studies
involve collecting data from groups of
clients who have been in the programme
for different amounts of time, and
comparing these groups. Usually this
involves comparing existing clients to
non-clients or new clients (see box 4 on
issues to consider). This approach is the
one used in the AIMS/SEEP impact
survey tool, as described in box 3.

enterprises and the communities in
which they live.

The AIMS/SEEP impact survey is not
an “off-the-shelf” tool but must be
adapted to each MFI’s needs and local
context. Some of the adaptations
made by
Imp-Act partners who
used this model as a starting point
include:

Switching the comparison group
from new clients to non-clients, or
clients waiting to be accepted onto
the programme.

Shifting the focus to put less
emphasis on enterprise-level change,
in favour of individual and household-
level.

Adding areas of enquiry, such as
spiritual indicators, or the impact of
training programmes.

CASE STUDY 2

At LAPO in Nigeria, an impact
survey chose to use waiting-list
clients as a comparison group.
Once the survey was under way,
however, the MFI realised that the
latest clients were much richer,
upon entry, than the existing
clients had been when they first
joined, so were not a good
comparison group. The situation
revealed the the need to target
clients more carefully.

At CAME in Mexico, the new
members they were hoping to use
as a comparison group turned out
to be mainly migrants from
different areas, and were therefore
poorer and more vulnerable than
the existing clients had been upon
entry. Here, as in LAPO, a quick
pilot study might have identified
these differences in advance.

Making Comparisons

The main point of having a comparison
group is to find out whether clients who
have received the programme’s services
have been able to improve their
situations compared to those who have
not. It is therefore necessary to choose
a comparison group who are as similar
as possible to the clients who are in the
programme.

One way of doing this is to use clients
who have just joined the programme
(new clients) but who have not yet
received the services (they are usually in
the stage of initial training). This makes
it easy to get the clients to respond.
However, it can be difficult to find a
suitable comparison group (see Case
study 2).

Another approach is to compare
clients with non-clients who are as
similar as possible. There are also
problems to be overcome with this
method: there might be fundamental
differences between those who choose
not to join a microfinance programme
and those who do. It might even be the
case that these non-clients are accessing

PAGE FOUR IMP-ACT PRACTICE NOTES NUMBER FOUR 2005



More intriguing information

1. Altruism and fairness in a public pension system
2. Nach der Einführung von Arbeitslosengeld II: deutlich mehr Verlierer als Gewinner unter den Hilfeempfängern
3. Bridging Micro- and Macro-Analyses of the EU Sugar Program: Methods and Insights
4. The name is absent
5. If our brains were simple, we would be too simple to understand them.
6. Banking Supervision in Integrated Financial Markets: Implications for the EU
7. The name is absent
8. INTERPERSONAL RELATIONS AND GROUP PROCESSES
9. The name is absent
10. The name is absent
11. HACCP AND MEAT AND POULTRY INSPECTION
12. An Investigation of transience upon mothers of primary-aged children and their school
13. The name is absent
14. The name is absent
15. The name is absent
16. Nietzsche, immortality, singularity and eternal recurrence1
17. Connectionism, Analogicity and Mental Content
18. Motivations, Values and Emotions: Three Sides of the same Coin
19. Environmental Regulation, Market Power and Price Discrimination in the Agricultural Chemical Industry
20. On Dictatorship, Economic Development and Stability