Fisher exact test and confidence interval2
Six Sigma – iSixSigma › Forums › Old Forums › General › Fisher exact test and confidence interval2
 This topic has 10 replies, 6 voices, and was last updated 17 years, 3 months ago by Frederico Zanqueta Poleto.

AuthorPosts

April 19, 2005 at 11:48 am #39065
I have the following problem
Before improvement: 12 defects out of 32 cases
After improvement: 1 defect out of 26 cases
I’m trying to prove the improvement
1. Confidence interval
Calculating the confidence intervals with the beta distribution approximating the binomial, the CIs do overlap, showing that the 2 proportions cannot be considered different
(0.161185, 0.500078) and (0.000973, 0.196370)
2. Fisher exact test
I this case I get: pvalue = 0.0155, showing that the proportions are different
Which of the 2 is the most reliable? I expected to get similar results
Thanks a lot
0April 20, 2005 at 7:45 am #118071Is there anybody who can help on this topic?
thanks
0April 20, 2005 at 8:41 am #118072Can you just collect more data and calculate the Z scores for the two cases and show the improvement?
VP0April 20, 2005 at 9:02 am #118073Sir , why dont you try chi square test to show the improvement.
make a contingency table and put it into minitab and hope that will solve the problem.0April 20, 2005 at 9:15 am #118075Joe,
the chisq test should not be used when the frequencies are very low (5 as a rule of thumb), and in my caso I have 1 out of 26.
My concern is about the method. I wish to know if the confidence intervals approach is statistically correct or not, and in that case which of the 2 is the most reliable: CI or Fisher (or Chisq)
thanks again
0April 20, 2005 at 9:18 am #118076when the frequency is less than 5 there is a correction factor which you can use but its not necessay that if frequency is less than 5 that you cannot use chi square test probably minitab uses the correction factor when calculating the expected frequencies.
Hope this helps!!0April 20, 2005 at 11:21 am #118085
facemanParticipant@faceman Include @faceman in your post and this person will
be notified via email.Fer,
How did you calculate the confidence limits? You are right to question a low p value when you have overlapping confidence limits. I would suspect that the method you used to calculate the confidence limits is less appropriate than Fisher’s exact test.
In general, I would trust Fisher’s exact. You have fairly small sample size relative to your p. Fisher’s is a good test in this case.0April 20, 2005 at 11:40 am #118092faceman,
thanks for your reply.
I calculated the confidence intervals using the beta distribution to approximate the binomial one. You can get the same results with a 1proportion test in Minitab
0April 20, 2005 at 12:03 pm #118095
facemanParticipant@faceman Include @faceman in your post and this person will
be notified via email.Fer,
I would definitely trust the Fisher’s exact method over the approximate method. If I remeber you had one treatment that an observed value of 1. That probably doesn’t approximate well. The exact method (hypergeometric) employed by Fisher’s is better in your case.
Regards,
faceman0April 20, 2005 at 4:29 pm #118126Fer
While I am generally unknown on this website, more of a lurker than a consistent poster, I thought I would jump in here and offer opinion. While your confidence interval calculation is acceptable (although slightly difficult and fraught with potential issues…such as the approximation itself), Fisher’s Exact test would be a very legitimate and sound alternative. If your original data had shown five defects in the after configuration, I am sure you would have went directly to Chi Squared. Since you only had 1 (and a much better improvement!), then Fisher’s was created for you!
Good luck with this!
Obiwan
P.S. Hello Darth…May 19 is coming…we will get to see your (ahem), birth!0May 2, 2005 at 11:47 am #118728
Frederico Zanqueta PoletoParticipant@FredericoZanquetaPoleto Include @FredericoZanquetaPoleto in your post and this person will
be notified via email.The book Categorical Data Analysis from Agresti (2002) discuss these intervals:
12 cases from 32
Lower bound Upper bound
Wald 0.2072630 0.5427370
score 0.2293389 0.5474559
Agresti 0.2288359 0.5479589
Clopper 0.2110003 0.5630775
Blaker 0.2175003 0.55367751 case from 26
Lower bound Upper bound
Wald 0.0354578003 0.1123809
score 0.0068219850 0.1889278
Agresti 0.0087373581 0.2044871
Clopper 0.0009732879 0.1963696
Blaker 0.0019732879 0.1828696The interval you showed for “1 case from 26” is a ClopperPearson interval. But I can not figure out what kind of interval formula you used for “12 cases from 32”. Note that even the Wald intervals, that are very bad for small samples, do not overlap. Neither the other methods.I suggest you to look in chapters 1 and 2 from Agresti. Although, in my opinion you can use any of these intervals, Fisher exact test is not a bad option.Sincerely,Fred0 
AuthorPosts
The forum ‘General’ is closed to new topics and replies.