The inability to separate out the treatment effect from selection bias is the identification
problem we are confronted with if we simply regress the outcome variable on the treatment
dummy. In this type of model, selection bias arises because the treatment variable is correlated
with unobserved characteristics. A natural solution therefore would seem to use proxy variables
of these unobservables in the outcome regression. Characterizing the problem in this way
suggests that many of the standard techniques that deal with endogenous regressors can be
used as potential solutions. However finding plausible ways of extracting exogenous variation
in treatment status in non-experimental settings often rests on a priori reasoning that might
be contestable or quite specific to some sub-population of the sample affected to participate in
the treatment as a result of the exogenous variable/s such models rely on. We return to these
non-experimental techniques in section 5.
3 Randomization
3.1 Motivation
Randomizing assignment to the treatment group (from a sample of potential participants in a
program) theoretically eliminates the confound from selection bias in estimates of mean impact.
Randomization involves a lottery process. Individuals from some well-defined population are
randomly selected into either the treatment group or the control group. An advantage of
this process is that it removes potential differences that could exist between the two groups for
which social scientists cannot control, or for which they find it difficult to control, such as ability,
work ethic, psychological disposition and so forth. Importantly, the observed and unobserved
attributes of individuals in the treatment and control groups prior to the intervention must be
independent of assignment to the treatment or control group. If this condition does not hold,
this will result in differences in mean outcomes ex-post that would falsely be attributed to the
intervention. However, when randomization is successfully implemented, the treatment effect
is unconfounded since treatment status is randomly allocated. 1
3.2 Internal Validity
Internal validity examines whether the specific design of an evaluation study generates reliable
estimates of counterfactual outcomes in a specific context (Ravallion, 2008). Despite the sim-
plicity involved in randomization, there may be a number of reasons why evaluation estimates
derived from this method lack internal validity. Bias may be introduced owing to selective
compliance with or attrition from the randomly assigned status. This occurs when individuals
assigned to the control group take deliberate action in order to attain the benefits of treatment.
For example, if an intervention is regionally based, or school based, individuals in the control
group may actively move schools or locations in order to be counted part of the treatment
group. Differential attrition in the treatment and control groups will also lead to biased esti-
mates. Since individuals who benefit from an intervention may be less likely to drop out of an
evaluation study than those who do not (control, group), this can result in differential attrition
between control and treatment groups. On the other hand, individuals randomly assigned to
the treatment group may choose not to comply with the treatment (for example, they may
neglect to take their pills, they may choose not to collect a social grant or to utilize a voucher
and so on), or, because they feel healthier, may stop complying with the requirements of the
1It is important bear in mind that random assignment in general does not “eliminate” selection bias because
participation is generally not open to all individuals in a population. Random assignment in such instances only
apply to a subset of the population. Under this sort of scenario, Heckman and Smith (1996) show that estimates
of mean impact will be unbiased because the effect of randomization is to balance the bias between the treated
and not-treated, so that the bias is differenced out when computing δ . However, when interest lies in some
other measure of central tendency, or higher order moments of the distribution of impacts, then randomisation
alone does not remove the effect of selection bias on estimates of impact. In this instance, combining social
experimentation with non-experimental methods of dealing with selection bias is a more appropriate strategy.