Designing a survey methodology
4 Assess changes in
clients’ status
The objective of an impact survey is to
be able to make statements about how
far changes in the lives of clients are
due to participation in a programme,
rather than to external factors. To
achieve this, your survey must allow
comparisons to be made either:
• Between those who use your MFI’s
services and those who do not (a
comparison group - similar to a control
group in experimental sciences). This is
known as a cross-sectional study
and/or
• Between the circumstances of clients
before they joined the programme
(provided by baseline data) compared to
their circumstances after they have used
the services for a period of time. This is
known as a longitudinal study.
Longitudinal studies usually make
comparisons over time - and have the
advantage of looking at changes in the
lives of the same clients. However, they
BOX 3: THE AIMS/SEEP IMPACT SURVEY TOOL
The AIMS/SEEP impact survey tool is
an example of a “middle-range”
survey tool, designed to look at the
types of issues MFIs may want to
investigate about impact, but at a
more modest cost than many
surveys. Its broad features are the
following:
• It is implemented at one point in
time rather than repeated over time.
• It asks the same questions of
clients who have been in the
programme for different periods of
time.
• It is used with clients who have just
entered the programme but not yet
received a loan as the main
comparison group.
• It collects information on four
aspects of clients’ lives: about them
as individuals; their households; their
require baseline information (i.e. from
before the client joined the programme),
which can then be compared with
information collected at a later date to
see what the changes are. This usually
takes a considerable amount of time
since starting to collect baseline
information now means that findings will
only be available after the clients are
again interviewed again at a future date
- usually one or two years.
Cross-sectional studies usually make
comparisons between different groups.
They are usually preferred by MFIs
because they can provide findings in a
shorter time frame. These studies
involve collecting data from groups of
clients who have been in the programme
for different amounts of time, and
comparing these groups. Usually this
involves comparing existing clients to
non-clients or new clients (see box 4 on
issues to consider). This approach is the
one used in the AIMS/SEEP impact
survey tool, as described in box 3.
enterprises and the communities in
which they live.
The AIMS/SEEP impact survey is not
an “off-the-shelf” tool but must be
adapted to each MFI’s needs and local
context. Some of the adaptations
made by Imp-Act partners who
used this model as a starting point
include:
• Switching the comparison group
from new clients to non-clients, or
clients waiting to be accepted onto
the programme.
• Shifting the focus to put less
emphasis on enterprise-level change,
in favour of individual and household-
level.
• Adding areas of enquiry, such as
spiritual indicators, or the impact of
training programmes.
CASE STUDY 2
At LAPO in Nigeria, an impact
survey chose to use waiting-list
clients as a comparison group.
Once the survey was under way,
however, the MFI realised that the
latest clients were much richer,
upon entry, than the existing
clients had been when they first
joined, so were not a good
comparison group. The situation
revealed the the need to target
clients more carefully.
At CAME in Mexico, the new
members they were hoping to use
as a comparison group turned out
to be mainly migrants from
different areas, and were therefore
poorer and more vulnerable than
the existing clients had been upon
entry. Here, as in LAPO, a quick
pilot study might have identified
these differences in advance.
Making Comparisons
The main point of having a comparison
group is to find out whether clients who
have received the programme’s services
have been able to improve their
situations compared to those who have
not. It is therefore necessary to choose
a comparison group who are as similar
as possible to the clients who are in the
programme.
One way of doing this is to use clients
who have just joined the programme
(new clients) but who have not yet
received the services (they are usually in
the stage of initial training). This makes
it easy to get the clients to respond.
However, it can be difficult to find a
suitable comparison group (see Case
study 2).
Another approach is to compare
clients with non-clients who are as
similar as possible. There are also
problems to be overcome with this
method: there might be fundamental
differences between those who choose
not to join a microfinance programme
and those who do. It might even be the
case that these non-clients are accessing
PAGE FOUR • IMP-ACT PRACTICE NOTES • NUMBER FOUR • 2005