fashion to imperfect experimental or quasi-experimental evaluations. We therefore treat this
approach separately from other well known non-experimental methods in order to motivate its
use as a solution to the problems of “internal validity” that often compromise otherwise well
designed randomized experiments. Section 5 deals with other non-experimental methods such
as IVs, double-differencing, and regression discontinuity. Even though the structure of the-
ses estimators are well known, we include a brief outline of them here since these approaches
are potentially quite useful in contexts where the assumptions underlying randomization or
matching do not hold.
Our second objective in this paper is to present a review of the existing evidence on the im-
pact of health interventions on individual welfare. The task of establishing the external validity
of health interventions has to be confronted irrespective of the the underlying methodology
used. Section 6 discusses some of the issues pertinent to external validity. This is followed in
section 7 by a review of how the range of available methods outlined in the preceding sections
has been employed in different settings and what is known about the impact of health inter-
ventions on individual productivity. Section 8 concludes the paper with an assessment of where
opportunities for further study might lie.
2 Identification of Program Impact
At the heart of the evaluation problem is the desire to know whether a particular program
or policy intervention has made a difference in the lives of those individuals or communities
affected by it, and if so, what the magnitude of this impact has been. In order to make this
kind of judgment, it is necessary to assess the welfare outcomes of program beneficiaries against
the counterfactual, namely:
(1) How would those people who benefited from an intervention have fared in the absence of
the intervention?
(2) How would those people who were not beneficiaries of an intervention have fared in the
presence of the intervention?
More specifically, consider the following hypothetical problem: let y1i refer to average out-
comes across all households in a given “community” i if the community has received some
health intervention, and let y0i refer to average outcomes across all households in this same
community i where no intervention took place. We are interested in what difference the receipt
of treatment has made to the average outcomes of households in this community; i.e., the dif-
ference y1i - y0i . The problem is that we will never have a given community both with and
without treatment at the same time.
Imagine that we have data on many communities, where some communities have received
treatment and others not. If we had this type of data, we could approximate y1i - y0i with
δ = E[y1i |T = 1] - E [y0i |T = 0]. This estimate, known as the single-difference estimate, is
confounded by the presence of selection bias. To see why this is so, imagine that we could
observe the counterfactual E [y0i |T = 1] - i.e., we can compute the average outcome of interest
across all households in non-beneficiary communities in an alternative state of the world in which
these communities were part of the beneficiary group. Now add and subtract this conditional
mean from the one used previously to give:
δ = E[y1i|T = 1] -E[y0i|T = 1]-E[y0i|T = 0] + E[y0i|T = 1]
'-----------------------------{z-----------------------------} '-----------------------------{z-----------------------------}
treatment effect selection bias
The first term in this expression is what we want to try to isolate: the effect of the intervention
on those that received it. We call this the treatment effect, or more precisely, the average
treatment effect on the treated (ATT). The last two terms together constitute selection bias
and picks up systematic unobservable differences between treatment and control households.