iSixSigma

Fisher exact test and confidence interval2

Six Sigma – iSixSigma Forums Old Forums General Fisher exact test and confidence interval2

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
    Posts
  • #39065

    Fer
    Participant

    I have the following problem
    Before improvement: 12 defects out of 32 cases
    After improvement: 1 defect out of 26 cases
    I’m trying to prove the improvement
    1. Confidence interval
    Calculating the confidence intervals with the beta distribution approximating the binomial, the CIs do overlap, showing that the 2 proportions cannot be considered different
    (0.161185, 0.500078) and (0.000973, 0.196370)
    2. Fisher exact test
    I this case I get: p-value = 0.0155, showing that the proportions are different
    Which of the 2 is the most reliable? I expected to get similar results
    Thanks a lot
     

    0
    #118071

    Fer
    Participant

    Is there anybody who can help on this topic?
    thanks
     

    0
    #118072

    VP
    Member

    Can you just collect more data and calculate the Z scores for the two cases and show the improvement?
    VP

    0
    #118073

    Whitehurst
    Participant

    Sir , why dont you try chi square test to show the improvement.
    make a contingency table and put it into minitab and hope that will solve the problem.

    0
    #118075

    Fer
    Participant

    Joe,
    the chi-sq test should not be used when the frequencies are very low (5 as a rule of thumb), and in my caso I have 1 out of 26.
    My concern is about the method. I wish to know if the confidence intervals approach is statistically correct or not, and in that case which of the 2 is the most reliable: CI or Fisher (or Chi-sq)
    thanks again
     

    0
    #118076

    Whitehurst
    Participant

    when the frequency is less than 5 there is a correction factor which you can use but its not necessay that if frequency is less than 5 that you cannot use chi square test probably minitab uses the correction factor when calculating the expected frequencies.
    Hope this helps!!

    0
    #118085

    faceman
    Participant

    Fer,
    How did you calculate the confidence limits?  You are right to question a low p value when you have overlapping confidence limits.  I would suspect that the method you used to calculate the confidence limits is less appropriate than Fisher’s exact test.
    In general, I would trust Fisher’s exact.  You have fairly small sample size relative to your p.  Fisher’s is a good test in this case. 

    0
    #118092

    Fer
    Participant

    faceman,
    thanks for your reply.
    I calculated the confidence intervals using the beta distribution to approximate the binomial one. You can get the same results with a 1-proportion test in Minitab
     

    0
    #118095

    faceman
    Participant

    Fer,
    I would definitely trust the Fisher’s exact method over the approximate method.  If I remeber you had one treatment that an observed value of 1.  That probably doesn’t approximate well.  The exact method (hypergeometric) employed by Fisher’s is better in your case.
    Regards,
    faceman

    0
    #118126

    Obiwan
    Participant

    Fer
    While I am generally unknown on this website, more of a lurker than a consistent poster, I thought I would jump in here and offer opinion.  While your confidence interval calculation is acceptable (although slightly difficult and fraught with potential issues…such as the approximation itself), Fisher’s Exact test would be a very legitimate and sound alternative.  If your original data had shown five defects in the after configuration, I am sure you would have went directly to Chi Squared.  Since you only had 1 (and a much better improvement!), then Fisher’s was created for you!
    Good luck with this!
    Obiwan
    P.S. Hello Darth…May 19 is coming…we will get to see your (ahem), birth!

    0
    #118728

    Frederico Zanqueta Poleto
    Participant

    The book Categorical Data Analysis from Agresti (2002) discuss these intervals:
    12 cases from 32
    Lower bound Upper bound
    Wald 0.2072630 0.5427370
    score 0.2293389 0.5474559
    Agresti 0.2288359 0.5479589
    Clopper 0.2110003 0.5630775
    Blaker 0.2175003 0.55367751 case from 26
    Lower bound Upper bound
    Wald -0.0354578003 0.1123809
    score 0.0068219850 0.1889278
    Agresti -0.0087373581 0.2044871
    Clopper 0.0009732879 0.1963696
    Blaker 0.0019732879 0.1828696The interval you showed for “1 case from 26” is a Clopper-Pearson interval. But I can not figure out what kind of interval formula you used for “12 cases from 32”. Note that even the Wald intervals, that are very bad for small samples, do not overlap. Neither the other methods.I suggest you to look in chapters 1 and 2 from Agresti. Although, in my opinion you can use any of these intervals, Fisher exact test is not a bad option.Sincerely,Fred

    0
Viewing 11 posts - 1 through 11 (of 11 total)

The forum ‘General’ is closed to new topics and replies.