Normality Tests and Sample Size
Six Sigma – iSixSigma › Forums › Old Forums › General › Normality Tests and Sample Size
 This topic has 12 replies, 8 voices, and was last updated 16 years, 10 months ago by DrSeuss.

AuthorPosts

February 26, 2005 at 8:41 pm #38559
When performing normality tests such as Anderson Darling and others, do small samples (1520) tend to fail to reject the null hypothesis that the data may have come from a normal distribution?
0February 27, 2005 at 2:03 am #115450It may be more of tdistro if continuous and < 30 – also depends on what's known and unknown. You also want to understand the effect you are looking for – Effect Size = Mean1 – Mean2 / s of either group (if variance is homogeneous). The greater the effect or difference between groups, the less power you will need, which is a function of sample size. In general, the larger the sample size the larger the power.
0March 1, 2005 at 12:21 pm #115528Ron,
You are correct…. Small samples sizes tend to “fail to reject” just as *very large* sample sizes tend to reject the null hypothesis…
Best Regards,
Bob J0March 7, 2005 at 11:17 am #115963The first point is that NO data come from any distribution. Data are generated by a process. A distribution (normal or otherwise) is a theoretical, abstract, mathematical concept, which is sometimes a useful model which allows us to estimate certain things about our data.
So called tests for normality (which are actually tests for nonnormality) enable us to see how useful our model is.
All of the formal tests for nonnormality are based around the idea of dividing the data up into classes (same as histogram frequencies) and then using Chisquare goodness of fit. The usual requirement is to have classes which contain a minimum 5 values each.
This means that, for n values, you will test the shape of the distribution for the middle (n10) values, and then check that there are 5 values in each tail. The tails will therefore be outside the range tested.
So for 20 values, you will test for lack of fit over the middle 50% of the assumed distribution only, and then check there are 5 values in each tail.
Almost any collection of 20 values will pass the test of nonnormality, but only out to +/ 0.67sigma. On the other hand, if you want to test for nonnormality further out, you need more data. For example, to test for nonnormality out to +/ 3sigma, you would need 3704 values, and to test out to +/ 6sigma you would need 5,049,883,388 values!
Since the there is no such thing as normally distributed data, and the normal model is only useful out to about +/ 3 sigma anyway, you will be almost certain to detect nonnormality beyond this range. This is why the traditional tables for hypothesis testing are usually given for significance levels of 0.1, 0.05, 0.01, 0.001 but no smaller.
Hope this makes the issue clearer.
Phil0March 7, 2005 at 1:56 pm #115967Let’s cut the mumbo jumbo.
Small sample sizes will almost always do this for you. Here’s a simple demonstration – put 7 data points that represent the uniform distribution into Minitab – and see what the normality test tells you. Similarly, if you have thousands of data points, the requirements for passing normality are extremely tight.
This is where some common sense comes in, ask yourself these 2 questions:
– Does the sample distribution look more or less normal?
– Based on what you know about the process, would you expect a highly nonnormal distribution?
Too many black belts extensively study the data, but forget to study the process that the data came from.0March 7, 2005 at 2:11 pm #115968Where did you see the mumbo jumbo in Phils response? It seemed more jumbo than mumbo and yours a tad light on the whys. Can you feel confident that offering a rather simplistic rules of the road perspective on data normality offers BBs the wherewithal to effectively use their statistical tools?
Vinny0March 7, 2005 at 2:14 pm #115969Hear, hear …
0March 7, 2005 at 2:15 pm #115970Well said …
It’s about time we heard more from people who actually know something about processing …0March 7, 2005 at 2:50 pm #115972I apologise if my post didn’t make sense to you. The key point is that the amount of the assumed distribution you are testing depends upon the amount of data you’ve got.
With a small amount of data you are not testing the tails (which is where distributions usually part company with reality) so you’ll be unlikely to detect nonnormality (or nonanything else) from just the middle of your data. With very large amounts of data you will be testing further and further out into the tails, and will be more and more likely to detect lack of normality (given that no data is actually normal).
It really depends on what you want to use your data for. If you are doing ANOVA etc, you will be fairly safe unless your data is detectably nonnormal with a small amount of data. However, if you want to say that any particular sigma level (beyond about 3) corresponds to any particular level of defectives, you will never have enough data to know the answer to within 2 or 3 orders of magnitude. The answer is based on your assumption!
To quote George Box “All models are wrong, but some models are useful” The normal distribution is a useful approximation up to about +/ 3 sigma.0March 7, 2005 at 3:03 pm #115976Glad to see you got my point about looking past the data to see the process.
0March 7, 2005 at 3:06 pm #115977I thought that was the specific point I was making in my first post (obviously not clearly enough)!
Glad to see that we are in “violent agreement”
Have a nice day.0March 7, 2005 at 3:12 pm #115980
Jonathon AndellParticipant@JonathonAndell Include @JonathonAndell in your post and this person will
be notified via email.For nearly every hypothesis test, it is easier to reject the null with larger sample sizes. By the way, if you have Minitab or a similar package, you might consider the probability plot. It “distors” the axes, so that the distribution in question looks like a straight line. It often is easier to decide whicxh distribution to use than AD or KS.
Also, never forget to plot the data in their time sequence, to see if there is some pattern. Neither AD nor probability plots can discern whether the underlying process was stable, or whether you are in fact plotting several processes on the chart.0March 7, 2005 at 9:30 pm #116027
DrSeussParticipant@DrSeuss Include @DrSeuss in your post and this person will
be notified via email.Ron,
Here is my experience with Minitab software, back in the dark ages of Minitab 10.5 – 12 eras, a company trainer indicated that running the Normality test on sample sizes less than 25 was not very productive. Seems that Mini would yield a big pvalue and you would assume you have normal data. It wasn’t until your sample size was as least 2030, would you really tap into the power of the test. Here is a thought, try running the AD normality test and compare those results to RyanJoiner and SmirnovKolmor….and if all yield the same results, then you should feel better about the call. Remember, you can always go back to basics, Chisquare and the old observed versus expected proportional relationships. Finally, here is the deal, if you have extremely small samples, your conclusions about any test will need to be validated with much larger samples. Remember, that is why they call it statistics.
0 
AuthorPosts
The forum ‘General’ is closed to new topics and replies.