The name is absent

. regress price r2 r3 r4 r5 length displ weight mpg

According to these estimates, cars with fair repair records cost an average of $907 more than cars with poor repair records.
The gap increases with each improvement in repair record. Cars with excellent repair records cost an average of $3,817 more
than cars with poor repair records.

The question may now arise: Which pairs of groups (categories of rep78) can we legitimately claim are different from each
other; which of these differences are unlikely to have arisen by chance? The answer hinges on what we view as “legitimate”.

The aggressive investigator might argue that groups 1 and 5 are different on the strength of the s statistic for the coefficient
on r5 (t = 2.135, with a p value of .037). The cautious investigator (or journal editor), however, would counter that many
comparisons of the different groups could have been made. Perhaps this test was selected for focus solely because it happens
to show a “significant” difference. And when multiple comparisons are made, the probability under the null of finding, say, a
t statistic as large as 2.135 is greater than .037. But how much greater is it—that is, what is the correct p value for this t statistic
when multiple comparisons are made?

There are many philosophical views on this problem. I examine the mechanics of one view—traditional adjustment for
multiple comparisons—in the context of regression-like models. (See [5s] oneway for a discussion of this approach in an ANOVA
context.) This view provides methods for making each test more conservative when there are multiple comparisons, so the overall
probability of making a Type I error for any pairwise comparison (declaring a difference significant when it is merely due to
chance) remains less than a predetermined value, such as 5 percent. We discuss three widely used approaches: the Bonferroni,
Sidak, and Scheffe tests.

The Bonferroni test is the simplest to implement. In this method, the cautious investigator would note that 10 pairs of
groups could have been compared and treat a reported p value of 0.037 as if it were 10 × 0.037 = 0.37. It would take a t value
of 2.9146 to be “significant” at the 5 percent level according to this logic. Using the Bonferroni rule, the contrast r5 vs. rl just
misses attaining significance.

The Sidak test is almost identical to the Bonferroni, unless the number of comparison groups is quite large. In our example,
the relevant critical value is about the same; a t statistic must be at least 2.9063 to be significant. The Scheffe test is even
more conservative, requiring a t statistic of 3.178. The Scheffe procedure is designed to hold for any linear combination of the
categories, not just for contrasts (comparisons of any two categories).

There is another consideration in a regression model that doesn’t arise in the one-way ANOVA context. In the ANOVA, the
means are guaranteed to be independent, since they come from independent samples. In the regression, a common adjustment
introduces correlation between the category means. The Scheffe method is a conservative answer that applies equally well in
the regression and ANOVA models. The Bonferroni and Sidak methods can become non-conservative in a regression context.

<<   <   13   14   15   16   17   18   19   >   >>