Sascha O. Becker and Marco Caliendo 9
odds of exposure to allopurinol but sensitive to a bias that would triple the odds. Our
example also highlights, that in some applications the significance level on the bounds
might fall first and than rise again. If we look, e.g. at the situation for Γ = 8, we get
a significance level p+mh of .0101 indicating a significant effect once again. It should be
clear, that this second significant value of p+mh indicates a significant negative treatment
effect. This is due to the fact, that we assume a large positive unobserved heterogeneity
which turns our previously significant positive treatment effect into a negative one.
5.2 The NSW Data Revisited
To illustrate mhbounds in a more common evaluation environment, we use the data also
used by Dehejia and Wahba (1999) and Smith and Todd (2005). It is well known that the
first study was very influential to promote matching as an evaluation method, whereas
the second one raised some doubts on the reliability of the results in non-experimental
evaluation settings.
The data come from Lalonde’s (1986) evaluation of non-experimental evaluation
methods and combines treated units from a randomized study of the National Supported
Work (NSW) training program with non-experimental comparison groups from surveys
as the Panel Study of Income Dynamics (PSID) or the Current Population Survey
(CPS).6 We restrict the sample to the experimental treatment group (n = 185) and
the PSID control group (n = 2490). The outcome of interest in DW99 are the post-
intervention real earnings in 1978 (RE78). Since we are interested in binary outcomes,
we define a new outcome variable employment taking the value of 1 if the individual
had positive real earnings in 1978 and 0 otherwise. The distribution of the outcome
variable is the following:
. tab employment d
|
employment |
d | ||
|
0 |
1 |
Total | |
|
0 |
286 |
45 |
331 |
|
1 |
2,204 |
140 |
2,344 |
|
Total |
2,490 |
185 |
2,675 |
To make the samples comparable we use propensity score matching by running
psmatch2 on the same specification as DW99.
. psmatch2 d age age2 education educ2 married black hispanic re74 re75 re742 re
> 752 blacku74, logit out(employment) noreplacement
Logistic regression Number of obs = 2675
LR chi2(12) = 935.35
Prob > chi2 = 0.0000
Log likelihood = -204.97537 Pseudo R2 = 0.6953
6. The data are available at Dehejia’s website: http://www.nber.org/~rdehejia/nswdata.html.