The name is absent

Sascha O. Becker and Marco Caliendo

Gamma : odds of differential assignment due to unobserved factors

Q_mh+ : Mantel-Haenszel statistic (assumption: over-estimation of treatment eff
> ect)

Q_mh- : Mantel-Haenszel statistic (assumption: under-estimation of treatment ef
> fect)

p_mh+ : significance level (assumption: over-estimation of treatment effect)
p_mh- : significance level (assumption: under-estimation of treatment effect)

Under the assumption of no hidden bias (Γ = 1), the QMH test-statistic gives a
similar result, indicating a significant treatment effect. The two bounds in the output
table can be interpreted in the following way: The Q⁺_MH statistic adjusts the MH
statistic downward for the case of positive (unobserved) selection. For the given example,
positive selection bias occurs when those most likely to participate tend to have higher
employment rates even in the absence of participation and given that they have the same
x-vector as the individuals in the comparison group. This leads to an upward bias in
the estimated treatment effects. The Q^-_MH statistic adjusts the MH statistic downward
for the case of negative (unobserved) selection. In other examples, the treatment effects
at Γ = 1 might be insignificant and the bounds tell us at which degree of unobserved
positive or negative selection the effect would become significant.

Given the positive estimated treatment effect, the bounds under the assumption that
we have under-estimated the true treatment effect (Q^-_MH) are somewhat less interesting.
The effect is significant under Γ = 1 and becomes even more significant for increasing
values of Γ if we have under-estimated the true treatment effect. However, looking at
the bounds under the assumption that we have over-estimated the treatment effect, i.e.
Q⁺_MH , reveals that already at relatively small levels of Γ, the result becomes insignificant.
To be more specific, with a value of Γ = 1.1 the result would not be significant at the
5%-significance level any more, with Γ = 1.15 it is even not significant at the 10%-
significance level, since the p-value is 0.109961. Clearly, based on these findings one
would be careful when interpreting the results.

However, it should be noted that these are worst-case scenarios. Hence, a critical
value of Γ = 1.15 does not mean that unobserved heterogeneity exists and that there
is no effect of treatment on the outcome variable. This result only states that the
confidence interval for the effect would include zero if an unobserved variable caused
the odds ratio of treatment assignment to differ between the treatment and comparison
groups by 1.15. One should keep in mind, that this test cannot directly justify the
unconfoundedness assumption. Hence, we cannot state whether the CIA does (not)
hold for the given setting (including inter alia the used data, the chosen covariates and
the specification of the propensity score). What we can say is, that the results are quite

<<   <   10   11   12   13   14   15   >   >>