Comparing Means

Six Sigma – iSixSigma Forums Old Forums General Comparing Means

Viewing 2 posts - 1 through 2 (of 2 total)
• Author
Posts
• #52849

lin
Participant

Hello,
I ran a one-way ANOVA and used Fisher’s LSD to compare the treatment means. I have 20 different levels of a factor in the ANOVA. The alpha value for the ANOVA was around 0.55 indicating no significant difference in the treatments means. However, when going through all the possible combinations, there are four pair of treatment means that are significantly different. The alpha value does not reflect that. Why is that? If I run Tukey for comparing treatment means, none are different.

0
#186451

Robert Butler
Participant

In point of fact the alpha value does reflect what you found.  Consider – you have 20 different levels and you ran all possible comparisons ( by this I assume you meant you ran all possible pairwise comparisons). This works out to 20!/((2!)*18!)) = 190 comparisons.
If I have a random chance of 5% of finding a significant difference and I test something 100 times then just by dumb luck five of those 100 tests should result in a “significant” difference. You managed to get 4 out of 190 with a P-value < .05 so your chance findings of significance didn't even equal 5%.
Given the number of tests, the actual level of significance needed for you to declare statistical significance for one of those 190 comparisons would have to be something like P < .00026.  It is this adjusted P-value that Tukey-Kramer is invoking and, as you discovered, none of the differences have a P-value of this magnitude.

0
Viewing 2 posts - 1 through 2 (of 2 total)

The forum ‘General’ is closed to new topics and replies.