Ttest help
Six Sigma – iSixSigma › Forums › Old Forums › General › Ttest help
 This topic has 17 replies, 9 voices, and was last updated 16 years ago by Anonymous.

AuthorPosts

May 16, 2006 at 1:39 am #43402
Two questions:How do I go about interpreting ttest data as it is provided by statistical software? I am using JMP and the results seem to be contradictory to what I am reading elsewhere. Also, I think I have read so many articles on ttest results that the whole null hypothesis/alternate hypothesis/t ratio things is now all the same…Is the ttest what I am after? I am comparing 2 processes to see if the new process is ‘better’ than an existing process. When I look at the 25 piece data, I can tell by the mean/stdev that the new process has a tighter stdev, with similar means, which tell sme it is better, right?Help me, please. Also, is there any book/article/text anyone would recommend for a rounded discussion on t, chi, f etc. testing and how to understand them?
0May 16, 2006 at 12:17 pm #137657When you have 1 factor (Process) and 2 levels (Process 1 and Process 2), you can use a oneway ANOVA or ttest and get the same outcome. The pvalues will be the same. If p is low, reject Ho or state that the means are different.
Both cases assume equal variances between the two groups. Do a test for equal variances using JMP, and see if Ho is rejected in this case. Most likey you will fail Ho, and hence violate the equal variance assumption. If you fail the equal variance assumption, the ttest or one way anova results are questionable. In your case, it seems like the variation is lower for the new process and you are more interested in this than a difference between mean values.
I would report on the equal variances test, and also report on Cpk for each group. (small data set, but JMP will give you confidence intervals on the Cpk values).
Good luck.
0May 16, 2006 at 1:05 pm #137661hacl:
Thanks for the help. I think I get it, but the whole logic behind failing Ho and how JMP expresses pvalues is confusing. When I run the ANOVA, I get this for the ttest:
Assuming equal variancesDifference
1.00667
t Ratio
4.204461
Std Err Dif
0.23943
DF
298
Upper CL Dif
1.47785
Prob > t
<.0001
Lower CL Dif
0.53548
Prob > t
<.0001
Confidence
0.95
Prob < t
1.0000
I presume that the “Prob” in the lower right hand corner is the pvalue, right? Which mean that Ho fails, and the means are significantly different…if I get that, I think I might be catching on.
Thanks for the help. Any good, simple to understand books you might be able to recommned?
0May 16, 2006 at 3:01 pm #137677
Ken FeldmanParticipant@Darth Include @Darth in your post and this person will
be notified via email.Unequal variances is NOT a major problem in either Minitab or JMP. You use the pooled s.d. or in JMP it will give you the results using Welch’s test. Normality of the individual data sets might be a concern but then you could easily use a nonparametric.
0May 16, 2006 at 3:13 pm #137679
luke skywalkerParticipant@lukeskywalker Include @lukeskywalker in your post and this person will
be notified via email.Well, sort of. First, Pete is on the right path in his desire to find a difference between the 2 processes. A good first step would be to check the normality of each sample. If either or both are not normal, a nonparametric test like Mood’s Median could work. If both samples pass a normality test then you can use either a 2sample ttest or a 1way ANOVA as suggested earlier.
Now the cool part. ANOVA does require (assume) equal variances in order to function and that’s of more concern than nonnormality. You can run the 2 sample ttest more easily, but need to check the variances first via an Ftest. Both JMP and Minitab can perform this. The reason you care is that you can select or deselect options in the ttest dialog to direct the program to calculate the tstatistic correctly based on whether the variances are equal. If the ARE equal, Darth is correct and a pooled standard deviation is used. If they ARE NOT equal, the individual variances are considered to provide a better test statistic. Either way, both JMP and Minnie can get you there with solid results.
Best of luck.0May 16, 2006 at 3:17 pm #137680
luke skywalkerParticipant@lukeskywalker Include @lukeskywalker in your post and this person will
be notified via email.Oh, and believe it or not, The Cartoon Guide to Statistics is a decent book for explaining this stuff in absorbable terms. Lots of stats books out there, just need to find the right themes. Psych stats and business stats books tend to provide clearer examples than engineering stats, but that will vary by individual. For me, cartoons work wonders as Darth can attest to.
0May 16, 2006 at 5:28 pm #137685
Ken FeldmanParticipant@Darth Include @Darth in your post and this person will
be notified via email.Dear Ringmaster Skywalker,
Good to hear from you. Is that why cartoons and comic books are core materials for the SS program down at the circus…..couldn’t resist, sorry.
Check this out…thought of you. Download is about half way down.
http://reignofthefallen.com/0May 16, 2006 at 6:59 pm #137690skywalker:
I think I am finally catching on, and I really appreciate the help. If I might continue with my questions…
With JMP I would test the normaility of the curve using the ‘distribution’ function for the sample, then go to ‘fit distribution’, and then perfom a ‘goodness of fit’ test (which in this case would be a ShapiroWilk), right?
If this is the case, I am really getting the hang of this, but am disappointed since our Quality Manager asked me to perform a ttest in order to qualify the new process. However, I noticed that the ttest passed, but the variation was vastly different between the two, which concerned me since I am making a statement of improvement of the process. Basically, I am thinking the QM is just spouting statistic phases without knowing how or why to apply them.
Regardless, all this information is invaluable to me. I sincerely appreciate the help.0May 16, 2006 at 8:27 pm #137691It basically comes down to what you’re looking to compare. If you’re testing for a difference in means, use the ttest (or ANOVA, etc.). If you’re performing a test to see if you’ve reduced variation, use the Ftest.
0May 16, 2006 at 11:10 pm #137693Pete,
Equal variance IS an assumption for ANOVA. As I mentioned earlier, do a test for equal variances. (Otherwise your ANOVA rejection region can be erroneous).
Your interest seems to be in the relative variation from one process as compared to the other, and not the difference in means. When doing the test for equal variances in JMP, you will get the various Ftest results (Levene, Bartlett, etc.), and will also see the Welch ANOVA results to compare means. Again, it seems as if you are not interested in means, but in how the variance is reduced with the new process.
Null Hypothesis: Variances are equal; Alpha = .05; test statistic = F ration (Levene, bartlett, etc.)
Statistical Principles and Methods by Johnson and Bhattacharyya is a good text. Also can try Statistical Methods and Data Analysis by Ott.
Good Luck.0May 22, 2006 at 8:09 am #137904The value of p you get from any hypothesis test is “the probability of getting a result at least as impressive as the result you got, IF the null hypothesis is correct”. For all hypothesis tests, the null hypothesis is not just the hypothesis of no difference, it is also the hypothesis of the validity of whatever model you are assuming. So for a ttest, you are looking at the probability of a difference in means at least as big as the one you got, given that there is no real difference AND that the data are NIID (normally, independently and identically distributed). Either a real difference, or a failure of assumptions, can result in a value smaller than the one you chose to regard as significant in advance of seeing the results. In the case of differences in variance between the two samples, there are variants of the ttest for unequal variance, which are approximations. The ttest is reasonably robust to minor deviations from normality, although the Ftest for equality of variances is not as robust. One of the key assumptions to check is that each of your samples have any sort of distribution at all. If either data group exhibits instability, then any assumption of a distribution (whether normal or anything else) will be invalid. Irrespective of the influence on the assumptions of the test itself, instability will invalidate any conclusions you may draw about the effect of any applied treatment (e.g. the change could have occured due to another cause entirely!) Have you tried putting the data onto a control chart? If there are differences between the two samples [either in terms of location (mean/median etc) or dispersion (standard deviation/variance/range etc)] then one or both sets of data will give out of control signals, relative to the control limits calculated for the other set. This method has several benefits:
it is distribution free – contrary to popular fiction about normal distributions, control chart limits are not based on distribution theory – so you don’t need to check for normality.
it will allow you to simultaneously check for differences in mean and variance, and
it will also allow you to check for consistency WITHIN each sample, to see if the process is changing independently of your modifications.0May 22, 2006 at 12:17 pm #137906
AnonymousParticipant@Anonymous Include @Anonymous in your post and this person will
be notified via email.it is distribution free – contrary to popular fiction about normal distributions, control chart limits are not based on distribution theory – so you don’t need to check for normality
I assume that you are a Wheelerite. As a Montgomeryite, I disagree with the above statement, especially if we are talking about individuals charts.
0May 22, 2006 at 5:58 pm #137917The answer of whether control charts are robust to normality issues or not is, It Depends. How precise do you want or need your control charts to be? I personally always use Individual and Moving Range charts together. If short term variation indicates an unstable process, the individual chart is of no use. If however the process is stable, the individuals chart will indicate reliable assignable cause limits. If you limit your tests to [1], exceeding the 3 sigma limit, the charts are very robust. If you need to use the 1 and 2 sigma limits for tests of control they are less robust.
0May 22, 2006 at 10:14 pm #137937
AnonymousParticipant@Anonymous Include @Anonymous in your post and this person will
be notified via email.Stability and normality are two separate issues. If you are using an Individuals Chart with nonnormal data, +/ 3 sigma limits, you will have high Type I and/or Type II errors.
This is easy to demonstrate via simulation. Run an individuals chart on an exponential distribution or strongly skewed distribution. The process “appears” to be unstable. Apply BoxCox. Rerun IMR chart on transformed data. Voila, a process that is stable and nonnormal.0May 24, 2006 at 8:34 am #138023Interesting stuff!
Actually I am a Shewhartian (at least as far as process control is concerned – in other uses of statistics I am a Bayesian), although Don Wheeler does seem to have a direct line of association from Shewhart via Deming, and this is distinctly different from Montgomery, Bissell, Ryan etc
I ran the simulation you suggested, using exponential data. (data generated in R using Mersenne Twister) I got three spurious “out of control signals” in 20 XmR simulations of 50 points each. The signals were all transient spikes (in other words they did not persist and did not therefore indicate a shift in the mean). Since all pseudorandom number generators will have some nonrandom properties (which will probably be detectable on a control chart), this is pretty good and compares favorably with theory.
However, the purpose of a control chart is to detect time related instability in process in order to provoke improvement, not to estimate the parameters of any assumed theoretical distribution. The key criteria used by Shewhart was whether the charts were economically optimised. Even if there were a slight increase in type I and type II error, the organisational cost of increasing the complexity of the charting outweighs the cost of the occasional false alarm. Hence “fine tuning” the charts by transformation of data is usually uneconomic, and divorces the tools from those best able to benefit from them, the guys working at the “sharp end”.
The key question is whether methods are effective. I find the use of control charts (without any form of restriction in terms of distributional assumptions) effective in reducing process variation (as did Japanese industry which, under the tutilage of Deming, acheived 18 – 25 sigma processes just using the “big 7” tools including control charts). I also find control charts useful in analysing the results of designed experiments, often in addition to more formal hypothesis tests. If your methods work for you, then that is fine too.
The point I was making was that ttests are relatively sensitive to their NIID assuptions (although not as much as Ftests), and that unless you have put observational data on a control chart, you will not know whether you have ANY single distribution, normal or otherwise, even though both formal and informal (i.e. graphical) tests for nonnormality may say otherwise. If the two groups of data you are comparing are not in themselves consistent, then can you in any meaningful sense be said to be comparing two processes?0May 24, 2006 at 3:33 pm #138034
Ken FeldmanParticipant@Darth Include @Darth in your post and this person will
be notified via email.Phil, great response, thanks.
Fellow Demingite0May 24, 2006 at 6:49 pm #138053
fsutculerParticipant@fsutculer Include @fsutculer in your post and this person will
be notified via email.Hello Pete,
I amfamiliar with doubth you got.
Sometimes I think that statistic makes life difficult.
In my opinion, if you see obvious difference in between groups/processes it is enough. So you can visualize the situation simply with histograms and comparing them.
Out of that, my favourite is another method which Shainin offered.
ýt is called as B vs. C . It is simply based on standard deviations comparisons.
If you give private email adress, I can send you some information about it.
Regards
fatih
0May 24, 2006 at 7:10 pm #138054Fatih,
Both your suppositions are incorrect. I’m sure if you take the time to ask a proper question – you’ll receive the correct answer.
Andy0 
AuthorPosts
The forum ‘General’ is closed to new topics and replies.