iSixSigma

T-test help

Six Sigma – iSixSigma Forums Old Forums General T-test help

  • This topic has 17 replies, 9 voices, and was last updated 16 years ago by Anonymous.
Viewing 18 posts - 1 through 18 (of 18 total)
  • Author
    Posts
  • #43402

    Ward
    Participant

    Two questions:How do I go about interpreting t-test data as it is provided by statistical software? I am using JMP and the results seem to be contradictory to what I am reading elsewhere. Also, I think I have read so many articles on t-test results that the whole null hypothesis/alternate hypothesis/t ratio things is now all the same…Is the t-test what I am after? I am comparing 2 processes to see if the new process is ‘better’ than an existing process. When I look at the 25 piece data, I can tell by the mean/stdev that the new process has a tighter stdev, with similar means, which tell sme it is better, right?Help me, please. Also, is there any book/article/text anyone would recommend for a rounded discussion on t-, chi-, f- etc. testing and how to understand them?

    0
    #137657

    Craig
    Participant

    When you have 1 factor (Process) and 2 levels (Process 1 and Process 2), you can use a one-way ANOVA or t-test and get the same outcome. The p-values will be the same. If p is low, reject Ho or state that the means are different.
    Both cases assume equal variances between the two groups.  Do a test for equal variances using JMP, and see if Ho is rejected in this case. Most likey you will fail Ho, and hence violate the equal variance assumption. If you fail the equal variance assumption, the t-test or one way anova results are questionable. In your case, it seems like the variation is lower for the new process and you are more interested in this than a difference between mean values.
    I would report on the equal variances test, and also report on Cpk for each group. (small data set, but JMP will give you confidence intervals on the Cpk values).
    Good luck.
     

    0
    #137661

    Ward
    Participant

    hacl:
    Thanks for the help. I think I get it, but the whole logic behind failing Ho and how JMP expresses p-values is confusing. When I run the ANOVA, I get this for the t-test:
    Assuming equal variances

    Difference

    1.00667

    t Ratio

    4.204461

    Std Err Dif

    0.23943

    DF

    298

    Upper CL Dif

    1.47785

    Prob > |t|

    <.0001

    Lower CL Dif

    0.53548

    Prob > t

    <.0001

    Confidence

    0.95

    Prob < t

    1.0000

     
     
     
     
    I presume that the “Prob” in the lower right hand corner is the p-value, right? Which mean that Ho fails, and the means are significantly different…if I get that, I think I might be catching on.
     
    Thanks for the help. Any good, simple to understand books you might be able to recommned?
     

    0
    #137677

    Ken Feldman
    Participant

    Unequal variances is NOT a major problem in either Minitab or JMP.  You use the pooled s.d. or in JMP it will give you the results using Welch’s test.  Normality of the individual data sets might be a concern but then you could easily use a nonparametric.

    0
    #137679

    luke skywalker
    Participant

    Well, sort of. First, Pete is on the right path in his desire to find a difference between the 2 processes. A good first step would be to check the normality of each sample. If either or both are not normal, a nonparametric test like Mood’s Median could work. If both samples pass a normality test then you can use either a 2-sample t-test or a 1-way ANOVA as suggested earlier.
    Now the cool part. ANOVA does require (assume) equal variances in order to function and that’s of more concern than non-normality. You can run the 2 sample t-test more easily, but need to check the variances first via an F-test. Both JMP and Minitab can perform this. The reason you care is that you can select or de-select options in the t-test dialog to direct the program to calculate the t-statistic correctly based on whether the variances are equal. If the ARE equal, Darth is correct and a pooled standard deviation is used. If they ARE NOT equal, the individual variances are considered to provide a better test statistic. Either way, both JMP and Minnie can get you there with solid results.
    Best of luck.

    0
    #137680

    luke skywalker
    Participant

    Oh, and believe it or not, The Cartoon Guide to Statistics is a decent book for explaining this stuff in absorbable terms. Lots of stats books out there, just need to find the right themes. Psych stats and business stats books tend to provide clearer examples than engineering stats, but that will vary by individual. For me, cartoons work wonders as Darth can attest to.

    0
    #137685

    Ken Feldman
    Participant

    Dear Ringmaster Skywalker,
    Good to hear from you.  Is that why cartoons and comic books are core materials for the SS program down at the circus…..couldn’t resist, sorry.
    Check this out…thought of you.  Download is about half way down.
    http://reignofthefallen.com/

    0
    #137690

    Ward
    Participant

    skywalker:
    I think I am finally catching on, and I really appreciate the help. If I might continue with my questions…
    With JMP I would test the normaility of the curve using the ‘distribution’ function for the sample, then go to ‘fit distribution’, and then perfom a ‘goodness of fit’ test (which in this case would be a Shapiro-Wilk), right?
    If this is the case, I am really getting the hang of this, but am disappointed since our Quality Manager asked me to perform a t-test in order to qualify the new process. However, I noticed that the t-test passed, but the variation was vastly different between the two, which concerned me since I am making a statement of improvement of the process. Basically, I am thinking the QM is just spouting statistic phases without knowing how or why to apply them.
    Regardless, all this information is invaluable to me. I sincerely appreciate the help.

    0
    #137691

    walden
    Participant

    It basically comes down to what you’re looking to compare. If you’re testing for a difference in means, use the t-test (or ANOVA, etc.). If you’re performing a test to see if you’ve reduced variation, use the F-test.

    0
    #137693

    Craig
    Participant

    Pete,
    Equal variance IS an assumption for ANOVA. As I mentioned earlier, do a test for equal variances.  (Otherwise your ANOVA rejection region can be erroneous).
    Your interest seems to be in the relative variation from one process as compared to the other, and not the difference in means. When doing the test for equal variances in JMP, you will get the various F-test results (Levene, Bartlett, etc.), and will also see the Welch ANOVA results to compare means. Again, it seems as if you are not interested in means, but in how the variance is reduced with the new process.
    Null Hypothesis: Variances are equal;  Alpha = .05; test statistic = F ration (Levene, bartlett, etc.)
    Statistical Principles and Methods by Johnson and Bhattacharyya is a good text. Also can try Statistical Methods and Data Analysis by Ott.
    Good Luck.

    0
    #137904

    Markert
    Participant

    The value of p you get from any hypothesis test is “the probability of getting a result at least as impressive as the result you got, IF the null hypothesis is correct”. For all hypothesis tests, the null hypothesis is not just the hypothesis of no difference, it is also the hypothesis of the validity of whatever model you are assuming. So for a t-test, you are looking at the probability of a difference in means at least as big as the one you got, given that there is no real difference AND that the data are NIID (normally, independently and identically distributed). Either a real difference, or a failure of assumptions, can result in a value smaller than the one you chose to regard as significant in advance of seeing the results. In the case of differences in variance between the two samples, there are variants of the t-test for unequal variance, which are approximations. The t-test is reasonably robust to minor deviations from normality, although the F-test for equality of variances is not as robust. One of the key assumptions to check is that each of your samples have any sort of distribution at all. If either data group exhibits instability, then any assumption of a distribution (whether normal or anything else) will be invalid. Irrespective of the influence on the assumptions of the test itself, instability will invalidate any conclusions you may draw about the effect of any applied treatment (e.g. the change could have occured due to another cause entirely!) Have you tried putting the data onto a control chart? If there are differences between the two samples [either in terms of location (mean/median etc) or dispersion (standard deviation/variance/range etc)] then one or both sets of data will give out of control signals, relative to the control limits calculated for the other set. This method has several benefits:

    it is distribution free – contrary to popular fiction about normal distributions, control chart limits are not based on distribution theory – so you don’t need to check for normality.
     it will allow you to simultaneously check for differences in mean and variance, and
    it will also allow you to check for consistency WITHIN each sample, to see if the process is changing independently of your modifications.

    0
    #137906

    Anonymous
    Participant

    it is distribution free – contrary to popular fiction about normal distributions, control chart limits are not based on distribution theory – so you don’t need to check for normality
    I assume that you are a Wheelerite. As a Montgomeryite, I disagree with the above statement, especially if we are talking about individuals charts. 
     

    0
    #137917

    Elbrin
    Participant

     The answer of whether control charts are robust to normality issues or not is, “It Depends.”  How precise do you want or need your control charts to be?  I personally always use Individual and Moving Range charts together.  If short term variation indicates an unstable process, the individual chart is of no use.  If however the process is stable, the individuals chart will indicate reliable assignable cause limits.  If you limit your tests to [1], exceeding the 3 sigma limit, the charts are very robust.  If you need to use the 1 and 2 sigma limits for tests of control they are less robust.

    0
    #137937

    Anonymous
    Participant

    Stability and normality are two separate issues.  If you are using an Individuals Chart with non-normal data, +/- 3 sigma limits, you will have high Type I  and/or Type II errors.
    This is easy to demonstrate via simulation.  Run an individuals chart on an exponential distribution or strongly skewed distribution.  The process “appears” to be unstable. Apply Box-Cox. Rerun IMR chart on transformed data. Voila, a process that is stable and non-normal.

    0
    #138023

    Markert
    Participant

    Interesting stuff!
    Actually I am a Shewhartian (at least as far as process control is concerned – in other uses of statistics I am a Bayesian), although Don Wheeler does seem to have a direct line of association from Shewhart via Deming, and this is distinctly different from Montgomery, Bissell, Ryan etc
    I ran the simulation you suggested, using exponential data. (data generated in R using Mersenne Twister) I got three spurious “out of control signals” in 20 XmR simulations of 50 points each. The signals were all transient spikes (in other words they did not persist and did not therefore indicate a shift in the mean). Since all pseudo-random number generators will have some non-random properties (which will probably be detectable on a control chart), this is pretty good and compares favorably with theory.
    However, the purpose of a control chart is to detect time related instability in process in order to provoke improvement, not to estimate the parameters of any assumed theoretical distribution. The key criteria used by Shewhart was whether the charts were economically optimised. Even if there were a slight increase in type I and type II error, the organisational cost of increasing the complexity of the charting outweighs the cost of the occasional false alarm. Hence “fine tuning” the charts by transformation of data is usually un-economic, and divorces the tools from those best able to benefit from them, the guys working at the “sharp end”.
    The key question is whether methods are effective. I find the use of control charts (without any form of restriction in terms of distributional assumptions) effective in reducing process variation (as did Japanese industry which, under the tutilage of Deming, acheived 18 – 25 sigma processes just using the “big 7” tools including control charts). I also find control charts useful in analysing the results of designed experiments, often in addition to more formal hypothesis tests. If your methods work for you, then that is fine too.
    The point I was making was that t-tests are relatively sensitive to their NIID assuptions (although not as much as F-tests), and that unless you have put observational data on a control chart, you will not know whether you have ANY single distribution, normal or otherwise, even though both formal and informal (i.e. graphical) tests for non-normality may say otherwise. If the two groups of data you are comparing are not in themselves consistent, then can you in any meaningful sense be said to be comparing two processes?

    0
    #138034

    Ken Feldman
    Participant

    Phil, great response, thanks.
    Fellow Demingite

    0
    #138053

    fsutculer
    Participant

    Hello Pete,
    I amfamiliar with doubth you got.
    Sometimes I think that statistic makes life difficult.
    In my opinion, if you see obvious difference in between groups/processes it is enough. So you can visualize the situation simply with histograms and comparing them.
    Out of that, my favourite is another method which Shainin offered.
    ýt is called   as B vs. C . It is simply based on standard deviations comparisons.
    If you give private e-mail adress, I can send you some information about it.
    Regards
    fatih
     

    0
    #138054

    Anonymous
    Guest

    Fatih,
    Both your suppositions are incorrect. I’m sure if you take the time to ask a proper question – you’ll receive the correct answer.
    Andy

    0
Viewing 18 posts - 1 through 18 (of 18 total)

The forum ‘General’ is closed to new topics and replies.