# Fisher exact test and confidence interval2

Six Sigma – iSixSigma Forums Old Forums General Fisher exact test and confidence interval2

Viewing 11 posts - 1 through 11 (of 11 total)
• Author
Posts
• #39065

Fer
Participant

I have the following problem
Before improvement: 12 defects out of 32 cases
After improvement: 1 defect out of 26 cases
I’m trying to prove the improvement
1. Confidence interval
Calculating the confidence intervals with the beta distribution approximating the binomial, the CIs do overlap, showing that the 2 proportions cannot be considered different
(0.161185, 0.500078) and (0.000973, 0.196370)
2. Fisher exact test
I this case I get: p-value = 0.0155, showing that the proportions are different
Which of the 2 is the most reliable? I expected to get similar results
Thanks a lot

0
#118071

Fer
Participant

Is there anybody who can help on this topic?
thanks

0
#118072

VP
Member

Can you just collect more data and calculate the Z scores for the two cases and show the improvement?
VP

0
#118073

Whitehurst
Participant

Sir , why dont you try chi square test to show the improvement.
make a contingency table and put it into minitab and hope that will solve the problem.

0
#118075

Fer
Participant

Joe,
the chi-sq test should not be used when the frequencies are very low (5 as a rule of thumb), and in my caso I have 1 out of 26.
My concern is about the method. I wish to know if the confidence intervals approach is statistically correct or not, and in that case which of the 2 is the most reliable: CI or Fisher (or Chi-sq)
thanks again

0
#118076

Whitehurst
Participant

when the frequency is less than 5 there is a correction factor which you can use but its not necessay that if frequency is less than 5 that you cannot use chi square test probably minitab uses the correction factor when calculating the expected frequencies.
Hope this helps!!

0
#118085

faceman
Participant

Fer,
How did you calculate the confidence limits?  You are right to question a low p value when you have overlapping confidence limits.  I would suspect that the method you used to calculate the confidence limits is less appropriate than Fisher’s exact test.
In general, I would trust Fisher’s exact.  You have fairly small sample size relative to your p.  Fisher’s is a good test in this case.

0
#118092

Fer
Participant

faceman,
I calculated the confidence intervals using the beta distribution to approximate the binomial one. You can get the same results with a 1-proportion test in Minitab

0
#118095

faceman
Participant

Fer,
I would definitely trust the Fisher’s exact method over the approximate method.  If I remeber you had one treatment that an observed value of 1.  That probably doesn’t approximate well.  The exact method (hypergeometric) employed by Fisher’s is better in your case.
Regards,
faceman

0
#118126

Obiwan
Participant

Fer
While I am generally unknown on this website, more of a lurker than a consistent poster, I thought I would jump in here and offer opinion.  While your confidence interval calculation is acceptable (although slightly difficult and fraught with potential issues…such as the approximation itself), Fisher’s Exact test would be a very legitimate and sound alternative.  If your original data had shown five defects in the after configuration, I am sure you would have went directly to Chi Squared.  Since you only had 1 (and a much better improvement!), then Fisher’s was created for you!
Good luck with this!
Obiwan
P.S. Hello Darth…May 19 is coming…we will get to see your (ahem), birth!

0
#118728

Frederico Zanqueta Poleto
Participant

The book Categorical Data Analysis from Agresti (2002) discuss these intervals:
12 cases from 32
Lower bound Upper bound
Wald 0.2072630 0.5427370
score 0.2293389 0.5474559
Agresti 0.2288359 0.5479589
Clopper 0.2110003 0.5630775
Blaker 0.2175003 0.55367751 case from 26
Lower bound Upper bound
Wald -0.0354578003 0.1123809
score 0.0068219850 0.1889278
Agresti -0.0087373581 0.2044871
Clopper 0.0009732879 0.1963696
Blaker 0.0019732879 0.1828696The interval you showed for “1 case from 26” is a Clopper-Pearson interval. But I can not figure out what kind of interval formula you used for “12 cases from 32”. Note that even the Wald intervals, that are very bad for small samples, do not overlap. Neither the other methods.I suggest you to look in chapters 1 and 2 from Agresti. Although, in my opinion you can use any of these intervals, Fisher exact test is not a bad option.Sincerely,Fred

0
Viewing 11 posts - 1 through 11 (of 11 total)

The forum ‘General’ is closed to new topics and replies.