iSixSigma

Help! Sample size Confidence

Six Sigma – iSixSigma Forums Old Forums General Help! Sample size Confidence

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
    Posts
  • #33507

    KavSS
    Participant

    Ok, say you have a manufacturing process that you’ve suddenly got reason to believe may have produced bad parts for the last month.  Purely hypothetical.  No, really.  Totally hypothetical.
    Anyway, the only way to check for the defect is a destructive test.  It produces 1000pts/day.
    Here’s the question:  It seems to me that if I destructively test, say, 100 parts out of the 1000, I should be able to make a statement like this: “I am 95% sure that the failure rate for all the parts is less than X%.”  Yet, I’m stuck on the math.  Brain embolism.  I can’t figure out how to make that statement.  Can someone please help?  Thank you!!
    -grant
     

    0
    #90711

    KavSS
    Participant

    Oh, and when I test the 100 parts, zero fail.

    0
    #90712

    Statman
    Member

    KavSS,
     
    Hopefully we can assume that the “day” in which the sample was drawn is representative of the month in which the defective parts were suspected (which is a big assumption) and that the 100 samples were drawn at random from the representative sampling frame.  If so, we would want to develop a 95% confidence interval for the resulting sample.  Since I am not in the mood to calculate the exact p-value from a binomial distribution by hand, I had Minitab perform a one sample Proportions test with the following result.
     
    Test and CI for One Proportion
     
    Test of p = 0.03 vs p < 0.03
     
                                                         Exact
    Sample      X      N  Sample p  95.0% Upper Bound  P-Value
    1           0    100  0.000000           0.029513    0.048
     
    You can therefore make the statement that based on the sample, “I am 95% sure that the failure rate for all the parts is less than 2.95%”.  So if there were 30,000 parts made in the month in question, as many as 885 defective based on the sample.  A larger sample would result in a tighter confidence interval.
     
    One last note, the 1000 per day has no bearing on the statistics.  Just because there is a finite number of parts made in a day, does not make this a finite population requiring a finite population adjustment.

    0
    #90752

    Gabriel
    Participant

    I am sending to iSixSigma a very simple Excel spreadsheet that helps calculate the confidence interval of proportions based on the binomial distribution. They will attach it here soon, so try again later if you dont see it.Confidence Interval Of Proportions Download (MS Excel, 18KB)By the way, the spreadsheet is pre-loaded with the data from your example which and, just by chance (?), shows the same result that the one provided by Statman using Minitab (defectives rate less than 2.95% with a 95% of confidence).Another interesting thing is that, based on the data of youir example, you can also say with a 95% of confidence that the defectives rate is greater than 0.05%, since 95% of the times you test 100 parts from a population with 0.05% of defectives you will find 0 defectives. Note that 0.05% in 1000 parts/day during 1 month gives you a total of 15 defectives.Putting both things together, you can say with a 90% of confidence that the actual defectives rate is between 0.05% an 2.95%.

    0
    #90765

    Statman
    Member

    Gabriel,
    Thank you for the spreadsheet.  It is very useful and very well done.  By the way, no “chance” involved with Minitab calculation as Minitab will calculate the exact cum. probabilites from a binomial in the 1-proportion test confidence interval as a default. 
    Interesting though, Minitab will always give zero as the lower bound for the confidence limit when the sample proportion is zero.  I would guess the logic to this is that the sample proportion should be within the confidence interval.  Your 90% confidence interval (0.05%, 2.95%) is the exact solution.  But try to convince a process owner that after his 100 unit sample found no defects that only 5 out of the next 100 days will have zero defects.  I think the “defectives rate less than 2.95% with a 95% of confidence” is easier to digest.
    Cheers
    Statman

    0
    #90781

    Gabriel
    Participant

    Thanks for your comments.
    The “chance” was just an ironic comment because of the poor fame that has Excel to handle statistical issues.
    About the confidence interval, yes, I also found very curious that the point estimator fell outside the confidence interval, and that’s in fact the reason why I put it in the post even when it was not directly related to the original question. I thought it would be an interesting curiosity.
    In fact, when I made the spreadsheet (I made it specifically as an answer to the original post) I tried both interval limits and was puzzled by the result of the lower limit. At first, I could not understand the logic of this result (the same would happen with any process owner, as you said). Then the following thoughts crossed my mind, and I would appreciate your comments on them (I see you are a “true” statman, I am just a stats user).
    – The binomial assumes an infinite population. In this context, p is continous and 0% and 100% are impossible values for p (just as any other exact value). Since there is no values in the domain of p below 0% and above 100%, thy can’t be limits of a confidence interval unless you make the trivial statements “I am 100% confident that p is above 0%” and “I am 100% confident that p is below 100%”, which hold true for any ammunt of defectives found in a sample of any size, so I don’t need a test to make those statements. Using 0% or 100% as interval limits for p would be like using +inifinite sigmas or -infinite sigmas as interval limits for µ.
    – The second thought is related to what i finnaly put in the post. A 90% two sided confidence interval is NOT the range where p will be with a 90% of probability given the result found in the sample. That’s what we (or I at least) tend to think, but it is wrong. We can’t say “There is a 90% of probability thet the true value of p is between 0.05% and 2.95% because I found 0 defectives in a sample of 100”. That’s why we use the word “confidence” instead of “probability”. In that wrong context, it is not logical to expect the point estimation to fall beyond the interval limits. The correct statement using the word “probability” would be “if the true p was the upper interval limit (2.95%) there would be a probability of 95% to find more than 0 defcetives in a sample of 100, and if the true p was the lower interval limit (0.05%) there would be a 95% of probability to find 0 or less defectives in a sample of 100” Three more things about this point: a) “0 or less” is trivial (it can not be less than zero) but analogous statements could be done, for example, for 2 defectives in the sample, in which case “2 or less” would be not trivial. b) By convention (I guess based on common sense and for the sake of balance) a two sided interval for a confidence (1-alpha)% is the combination of a lower and an upper confidence interval of confidence (1-alpha/2)% each. It would be possible (would it be?) to make other intervals with the same confidence (for example, using (1-alpha/3) and (1-2*alpha/3)). The extreme case is the one sided confidence interval, wich can bi thought as a two sided interval made of the combination of a (1-alpha) and a (1-alpha/infinite=1) one sided intervals. In this way, it could be said that (0; 2.95%) is a two sided 95% confidence interval (a combination of a “below 2.95%” 95% confidence interval and a “above 0%” 100 %confidence interval.
    – Even whn it’s hard to explain it to a process owner in terms of a confidence interval for defectives rate, it would be eassy to explain the same mathematical soliution under the perspective of another application. Think of this test as it was an acceptance sampling plan where the sample size n is 100 and the acceptance number c is 0. In such a plan, for an AQL of 0.05% you would have a producer’s risk (alpha) of 5%, meaning that you would still reject batches with 0.05% of defectives 5% of the times. In the same way, for a RQL of 2.95% you would have a consumer’s risk (beta) of 5%, meaning that you would still fail to reject batches with 2.95% of defectives 5% of the times. That would be easy to understand by many process owners, and probably would trigger the question “Is the information from the result of this test enough to make a final decision?” Of course the same question would be triggered by the statement “After testing 100 parts and finding no defect I can say, with a 95% of confidence, that the defectives rate is no more than 2.95%” (no lower limit). The interesting thing of adding a lower limit (or using the samplin plan analogy) is that it puts clear not only that the defectives rate could be a somehow haigh value, but also that I am almost sure that if the defcives rate was a somehow low value I would not detect it with this test.

    0
    #90794

    Ranjan
    Participant

    Statman, Is there any reason why you took the test proportion to be 3%? Just curious.
    Rahul

    0
    #90796

    Statman
    Member

    Gabriel,
     
    You appear to be much more than just a user of statistics.  The high level of thought into this discussion, your work on the spreadsheet, as well as other thorough responses you have posted proves otherwise.  Of course we are all just humble students of the discipline; even those that call themselves masters, experts, and consultants (some, including myself, lacking the humility part at times).
     
    Somewhere in the teaching of inferential statistics an important aspect of the definition of confidence intervals got lost.  In most of the six-sigma training material that I have seen the definition of a confidence interval is “We are 95% confident that the true population parameter is within this interval”.  This definition, albeit much easier to get across to students, is not really correct.  As you know C.I. are based in the aspect of the central limit theorem that If we select a random sample of size n from a defined population, we can draw inferences about all samples of size n that will be drawn in the future from the same population.  This means that the CLT allows us to produce the theoretical distribution of all possible samples from that population.  Notice that this is about the sample parameter and not the true population parameter.
     
    So what is the correct statement about a confidence interval?  What a 95% confidence interval implies is that if we the sample x number of  times, 95% of those samples will have a sample parameter within the confidence interval of the original sample.  Therefore, what a confidence interval is about is the risk of a different conclusion being drawn from a future sample of the same population.  Confidence intervals are statements about the sample parameter not the true parameter.  A leap of faith is involved to get to the true parameter.  This is why 0% or 100% can be a possible bound in a confidence interval for a sample proportion as they are obviously possible outcomes of a future sample.
     
    What you calculated with your interval of (0.05%, 2.95%) is the range of a binomial distribution with p=0, n=100 that cuts of 5% of the distribution in each of the tails.   
     
    I am not quite sure what you meant by alpha/3 etc.  Just remember that alpha is the risk in drawing a wrong conclusion.  The alpha/2 is just dividing the risk equally into the two tails of the distribution.
     
    Hope this helps,
     
    Cheers
     
    Statman

    0
    #90798

    Statman
    Member

    Rahul,
    I guessed that the upper bound would be close to 3% just to get the Minitab program to work knowing that Minitab would produce the upper 95% CI if I did a one sided test.  I could have used any percent as an initial guess. I just wanted the confidence interval.
     

    0
    #90800

    Gabriel
    Participant

    No, really, I am just a user.
    What is very strong in my case (and maybe not so strong for some others) is that I NEED to understand the why, not only the how, so when the trainer says something like “To calculate the control limits use Xbarbar±A2*Rbar” there my questions start. I just refuse to apply tools just because “belive me, it works”. This is may way of explaining the things too (as you probably noted) and that’s what make my posts so long. My answers are seldom like “Do this”.
    Returning to the CIs, my understanding clearly differs from what you explain. In line whith what I’ve just said above, I need to know wether you are wrong, I missunderstood your explanation, or I am wrong and, if the last is true, fix it. However, I think that the conversation may go beyond the scope of interest of most users of this forum. Would you contact me off-line? [email protected]

    0
    #90802

    KavSS
    Participant

    Thanks to all for the great discussion, and especially to Gabriel for the spreadsheet.  Very, very helpful, and much appreciated!

    0
Viewing 11 posts - 1 through 11 (of 11 total)

The forum ‘General’ is closed to new topics and replies.