iSixSigma

Comparing Means

Six Sigma – iSixSigma Forums Old Forums General Comparing Means

This topic contains 1 reply, has 2 voices, and was last updated by  Robert Butler 9 years, 8 months ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #52849

    lin
    Participant

    Hello,
    I ran a one-way ANOVA and used Fisher’s LSD to compare the treatment means. I have 20 different levels of a factor in the ANOVA. The alpha value for the ANOVA was around 0.55 indicating no significant difference in the treatments means. However, when going through all the possible combinations, there are four pair of treatment means that are significantly different. The alpha value does not reflect that. Why is that? If I run Tukey for comparing treatment means, none are different.

    0
    #186451

    Robert Butler
    Participant

     In point of fact the alpha value does reflect what you found.  Consider – you have 20 different levels and you ran all possible comparisons ( by this I assume you meant you ran all possible pairwise comparisons). This works out to 20!/((2!)*18!)) = 190 comparisons. 
      If I have a random chance of 5% of finding a significant difference and I test something 100 times then just by dumb luck five of those 100 tests should result in a “significant” difference. You managed to get 4 out of 190 with a P-value < .05 so your chance findings of significance didn't even equal 5%. 
      Given the number of tests, the actual level of significance needed for you to declare statistical significance for one of those 190 comparisons would have to be something like P < .00026.  It is this adjusted P-value that Tukey-Kramer is invoking and, as you discovered, none of the differences have a P-value of this magnitude.

    0
Viewing 2 posts - 1 through 2 (of 2 total)

The forum ‘General’ is closed to new topics and replies.