iSixSigma

Normality Tests and Sample Size

Six Sigma – iSixSigma Forums Old Forums General Normality Tests and Sample Size

Viewing 13 posts - 1 through 13 (of 13 total)
  • Author
    Posts
  • #38559

    Ron
    Member

    When performing normality tests such as Anderson Darling and others, do small samples (15-20) tend to fail to reject the null hypothesis that the data may have come from a normal distribution?

    0
    #115450

    MP
    Participant

    It may be more of t-distro if continuous and < 30 – also depends on what's known and unknown. You also want to understand the effect you are looking for – Effect Size = Mean1 – Mean2 / s of either group (if variance is homogeneous). The greater the effect or difference between groups, the less power you will need, which is a function of sample size. In general, the larger the sample size the larger the power.

    0
    #115528

    Bob J
    Participant

    Ron,
    You are correct….  Small samples sizes tend to “fail to reject” just as *very large* sample sizes tend to reject the null hypothesis… 
    Best Regards,
    Bob J

    0
    #115963

    Markert
    Participant

    The first point is that NO data come from any distribution. Data are generated by a process. A distribution (normal or otherwise) is a theoretical, abstract, mathematical concept, which is sometimes a useful model which allows us to estimate certain things about our data.
    So called tests for normality (which are actually tests for non-normality) enable us to see how useful our model is.
    All of the formal tests for non-normality are based around the idea of dividing the data up into classes (same as histogram frequencies) and then using Chi-square goodness of fit. The usual requirement is to have classes which contain a minimum 5 values each.
    This means that, for n values, you will test the shape of the distribution for the middle (n-10) values, and then check that there are 5 values in each tail. The tails will therefore be outside the range tested.
    So for 20 values, you will test for lack of fit over the middle 50% of the assumed distribution only, and then check there are 5 values in each tail.
    Almost any collection of 20 values will pass the test of non-normality, but only out to +/- 0.67sigma. On the other hand, if you want to test for non-normality further out, you need more data. For example, to test for non-normality out to +/- 3sigma, you would need 3704 values, and to test out to +/- 6sigma you would need 5,049,883,388 values!
    Since the there is no such thing as normally distributed data, and the normal model is only useful out to about +/- 3 sigma anyway, you will be almost certain to detect non-normality beyond this range. This is why the traditional tables for hypothesis testing are usually given for significance levels of 0.1, 0.05, 0.01, 0.001 but no smaller.
    Hope this makes the issue clearer.
    Phil

    0
    #115967

    Kado
    Participant

    Let’s cut the mumbo jumbo.
    Small sample sizes will almost always do this for you.  Here’s a simple demonstration – put 7 data points that represent the uniform distribution into Minitab – and see what the normality test tells you.  Similarly, if you have thousands of data points, the requirements for passing normality are extremely tight.
    This is where some common sense comes in, ask yourself these 2 questions:
    – Does the sample distribution look more or less normal?
    – Based on what you know about the process, would you expect a highly non-normal distribution?
    Too many black belts extensively study the data, but forget to study the process that the data came from.

    0
    #115968

    Dayton
    Member

    Where did you see the mumbo jumbo in Phil’s response?   It seemed more jumbo than mumbo and yours a tad light on the “whys”.   Can you feel confident that offering a rather simplistic “rules of the road” perspective on data normality offers BB’s the wherewithal to effectively use their statistical tools?
     Vinny

    0
    #115969

    Anonymous
    Guest

    Hear, hear …

    0
    #115970

    Anonymous
    Guest

    Well said …
    It’s about time we heard more from people who actually know something about processing …

    0
    #115972

    Markert
    Participant

    I apologise if my post didn’t make sense to you. The key point is that the amount of the assumed distribution you are testing depends upon the amount of data you’ve got.
    With a small amount of data you are not testing the tails (which is where distributions usually part company with reality) so you’ll be unlikely to detect non-normality (or non-anything else) from just the middle of your data. With very large amounts of data you will be testing further and further out into the tails, and will be more and more likely to detect lack of normality (given that no data is actually normal).
    It really depends on what you want to use your data for. If you are doing ANOVA etc, you will be fairly safe unless your data is detectably non-normal with a small amount of data. However, if you want to say that any particular sigma level (beyond about 3) corresponds to any particular level of defectives, you will never have enough data to know the answer to within 2 or 3 orders of magnitude. The answer is based on your assumption!
    To quote George Box “All models are wrong, but some models are useful” The normal distribution is a useful approximation up to about +/- 3 sigma.

    0
    #115976

    Kado
    Participant

    Glad to see you got my point about looking past the data to see the process.

    0
    #115977

    Markert
    Participant

    I thought that was the specific point I was making in my first post (obviously not clearly enough)!
    Glad to see that we are in “violent agreement”
    Have a nice day.

    0
    #115980

    Jonathon Andell
    Participant

    For nearly every hypothesis test, it is easier to reject the null with larger sample sizes. By the way, if you have Minitab or a similar package, you might consider the probability plot. It “distors” the axes, so that the distribution in question looks like a straight line. It often is easier to decide whicxh distribution to use than A-D or K-S.
    Also, never forget to plot the data in their time sequence, to see if there is some pattern. Neither A-D nor probability plots can discern whether the underlying process was stable, or whether you are in fact plotting several processes on the chart.

    0
    #116027

    DrSeuss
    Participant

    Ron,
    Here is my experience with Minitab software, back in the dark ages of Minitab 10.5 – 12 eras, a company trainer indicated that running the Normality test on sample sizes less than 25 was not very productive.  Seems that Mini would yield a big p-value and you would assume you have normal data.  It wasn’t until your sample size was as least 20-30, would you really tap into the power of the test.  Here is a thought, try running the AD normality test and compare those results to Ryan-Joiner and Smirnov-Kolmor….and if all yield the same results, then you should feel better about the call.  Remember, you can always go back to basics, Chi-square and the old observed versus expected proportional relationships.  Finally, here is the deal, if you have extremely small samples, your conclusions about any test will need to be validated with much larger samples.  Remember, that is why they call it statistics.
     

    0
Viewing 13 posts - 1 through 13 (of 13 total)

The forum ‘General’ is closed to new topics and replies.