iSixSigma

Attribute Gage RR Question from a newby

Six Sigma – iSixSigma Forums Old Forums General Attribute Gage RR Question from a newby

Viewing 18 posts - 1 through 18 (of 18 total)
  • Author
    Posts
  • #34673

    Newby
    Participant

    Hi all,
    I would appreciate any insight anyone can give on this. I am trying to get my Yellow Belt and completed an Attribute Gage R&R. We did 3 trials with 3 Operators with 6 bad parts and 14 good parts as the standard. 2 appraisers agreed with themselves 19 out 20 times and one appraiser agreed 20 out of 20 times. 2 appraisers agreed with the standard 19 times out of 20 with the standard and one appraiser agreed 20 out of 20 times with the standard ending up with between appraisers 20 inspeacted and 18 matched for 90% and all appraisers vs standard at 90%. To me this looked like a successful gage R&R however:
    Our Quality Guru (who by the way is not an MBB or BB but claims to be at their level) advised that we need a do-over on our Gage R&R because we only have 20 samples and the difference between our lower and upper control limit (according to him) indicates that although on this run it matched well, there is a 95% lower limit of 75.1 % that telles him the operator could be as bad as 75% efficient and that we do not have a large enough sample size for attribute data to have faith in our Gage R&R.
    Is he yanking our chain or does he know what he is talking about?
    Thanks for any light you can shed on this.
    Newby
     

    0
    #95887

    Justin Swain
    Participant

    He is correct. You should take a min. of 30 samples (15 good, 15 bad, depending on the cost to make bad parts). By using this sample size you should see a higher confidence interval (I think this is what you are refering to wrt the control limit), with a min of 80% for a pass.
    I hope this helps.
    BBHelper

    0
    #95899

    N14diesel
    Participant

    Your “guru” is correct in that as your sample size increases, the range of your 95% confidence interval shrinks.  Although the results of your study indicate 90% agreement with the standard you have established, your sample size of 20 parts is only large enough to assure you that the true % match with the standard is between 75% and 100%.

    0
    #95902

    Statman
    Member

    Newby,
     
    Your Quality Guru is full of it.
     
    As is often the case, the Guru is misinterpreting the concept of confidence intervals.  All values within a confidence interval do not have equal likelihood of occurrence.   There is not a uniform distribution of probability across the CI.  A better way to state the question is:  given the true proportion of agreement between the appraiser and the standard is 75%, what is the probability that on a given evaluation the appraiser would score 19 or better out of 20?  The answer to that from excel is BINOMDIST(19,20,0.25,TRUE)= 0.0243.  So there is only a 2.43% chance that the appraiser is at a 75% agreement level and will score 19 out of 20 or better.
     
    If you had run 30 samples, there is still a chance, albeit smaller, that the true proportion of agreement is 75% with 29 out of 30. It is BINOMDIST (29, 30, 0.25, TRUE) = 0.002.  The table below lists the probability of the result (19 out of 20, 29 out of 30, and 28 out of 30) for various assumed true proportions of agreement (P.null).  I have also listed the resulting 95% lower CI for each result.  If you were to run 30 trails rather than 20 with 29 matches, the lower limit would only increase to 82%.  If there was one additional failure, the lower limit would have been 78%.  I don’t know what your gurus interpretation of an acceptable lower limit would be but if you have one mismatch, you would have to have 55 trials for the 95% lower CI to be greater than 90%.  This, of course, makes the whole intent of the study trivial.  As soon as you have one mismatch you might as well stop and declare the inspection system flawed.
     
    P.null      Matches   Trials      Probability      95% lower CI
    0.75                19        20          0.0243             0.751
    0.80                19        20          0.0692             0.751
    0.85                19        20          0.1756             0.751
    0.90                19        20          0.3917             0.751
    0.95                19        20          0.7358             0.751
    0.75                29        30          0.0020             0.823
    0.80                29        30          0.0105             0.823
    0.85                29        30          0.0480             0.823
    0.90                29        30          0.1837             0.823
    0.95                29        30          0.5535             0.823
    0.75                28        30          0.0020             0.779
    0.80                28        30          0.0442             0.779
    0.85                28        30          0.1514             0.779
    0.90                28        30          0.4114             0.779
    0.95                28        30          0.8122             0.779
     
    A better measure of the reliability of the inspection system is the Kappa coefficient.  The Kappa coefficient is the proportion of agreement between raters and the standard after agreement by chance has been removed.  Minitab has the option to give you Kappa values.  As a general rule, if Kappa is lower than 0.7, the measurement system needs attention. The problems are almost always caused by either an ambiguous operational definition or a poorly trained rater. Kappa coefficients above 0.9 are considered excellent, and there is rarely a need to try to improve beyond this level.  I have re-evaluated your data based on the information you have provided and got the following results for Kappa:
     
    Within Appraiser
    Appraiser Response    Kappa SE Kappa        Z P(vs > 0)
    1         f          1.0000   0.2236   4.4721     0.000
    2         f          0.8746   0.2236   3.9114     0.000
    3         f          0.8746   0.2236   3.9114     0.000
    Each Appraiser vs Standard
    Appraiser Response    Kappa SE Kappa        Z P(vs > 0)
    1         f          1.0000   0.1581   6.3246     0.000
    2         f          0.9373   0.1581   5.9280     0.000
    3         f          0.9373   0.1581   5.9280     0.000
    Between Appraisers
    Response    Kappa SE Kappa         Z P(vs > 0)
    f          0.9179   0.0577   15.8989     0.000
    All Appraisers vs Standard
    Response    Kappa SE Kappa         Z P(vs > 0)
    f          0.9582   0.0913   10.4966     0.000
    As you can see, the only concern is the within appraiser Kappa level.
     
    You asked, “Is he yanking our chain or does he know what he is talking about?”  In my opinion he is doing worse than yanking your chain.  He is acting in the worse way that a coach/expert can act; be it that the coach is a MBB, BB, or Guru.  He is playing data/method policeman and not providing any value added help to your project.  Disregarding the work that you have done just because it does not meet some semi-arbitrary sample size requirement is not value added.  In one book on attribute MSA I looked some time ago had a stated 30 part requirement and then proceeded to show an example with 20 parts.  I guess the rules only apply to the practitioners.
     
    The questions he should be asking of these results and having you investigate are:
    Ø      What is the inspection used for?
    Ø      How critical is a failed good part?
    Ø      How critical is a pass bad part?
    Ø      Were the mismatches that 2 of the 3 appraisers experienced on the same sample?
    Ø      Were the mismatches passed bad parts or failed good parts?
    Ø      If they were passed bad parts, were the parts close to standard?
    Ø      If they were failed good parts, were the parts borderline acceptable?
     
    Statman

    0
    #95905

    Newby
    Participant

    Statman thank you.  Where can I find how to calculate Kappa in Minitab? I looked all over the gage R&R and the help file and can’t find it.
    Newby

    0
    #96394

    chngagent
    Participant

    very good explanation!

    0
    #96404

    Joe Sener
    Participant

    Statman,
    This is the most lucid description I have seen. Could you share the text book source on Attribute gage R&R?
    One of my real frustrations is with people who obfuscate rather than illuminate.
    Joe Sener

    0
    #96405

    chngagent
    Participant

    joe,
    a lot of the information stat posted can be found in minitab. look in your help files for kappa statistic.
    hope this helps.

    0
    #96410

    xuvman
    Member

    what’s ob ..ob …obfu  obscufate mean>?

    0
    #96416

    Joe Sener
    Participant

    to make obscure!

    0
    #96418

    xuvman
    Member

    Sorry Joe, I  just couldnt resist a smart-aleck urge I get every now and then!!    by they way an excellent discussion on R&R with attributes

    0
    #96479

    Statman
    Member

    Joe,
     
    Thank you for the compliment on my post.
     
    I have yet to see a good reference for attribute Gage R&R that includes a discussion of Cohen’s kappa coefficient.  Although I have not spent anytime looking for one so there may be one out there.  A brief but very good discussion of Kappa is in the following Quality Progress article:
     
    D. Futrell (1995). “When Quality is a Matter of Taste, Use Reliability Indexes,” Quality Progress, 28 (5), 81-86
     
    I tend to look at an application of a method and try to understand the underlying statistical/mathematical methodology and assumptions then search for the reference source.  In most cases, the method is nothing more than the application of a fundamental inferential statistical concept that has been renamed to give the author credit or association to the application.  After all, there are only four inferential (analytical) applications; all the thousands of methods are just variations on those four themes for various situations.
     
    Most of the in depth work in qualitative measurement system analysis has been in the field of Psychometrics not industrial statistics. 
     
    Cheers,
     
    Statman

    0
    #96523

    Jonathon L. Andell
    Participant

    Statistics aside, my experience shows that getting 19 out of 20 to agree represents unusually good inspection practice. I know some folks that beat themselves up just to achieve 15 out of 20. Starting below 50% agreement is rather common.
    By the way, Statman and Joe Sener, keep up the competent and enjoyable dialogue. Even a thick-skulled Neanderthal like me can learn something now and then…

    0
    #96529

    Deelip Wasadikar
    Participant

    Dear XUVMAN,
    AIAG’s 3rd edition of MSA manual gives the various approaches.
    You may use any one of them,
    Thanks
    D.P.Wasadikar

    0
    #96543

    SSBB
    Member

    Looks like I am struggling with my posts.
    Here are my 2 questions.
    First I tried getting the values you have in the table. The formula BINOMDIST in excel doesn’t seem to work for me after typing the same values you have given.
    Maybe, I am ignorant, but still lots to learn. Hope you clear the confusion.
    P.null  Matches  Trails  prob(given) prob (found)  95% LCL  prob.(using P.null,FALSE)
    0.75     19            20       0.0243        1.0000                 ?              0.0211
    0.8       19            20       0.0692        1.0000                 ?              0.0576
    0.85     19            20       0.1756        1.0000                 ?              0.1368
    0.9       19            20       0.3917        1.0000                 ?              0.2702
    0.95     19            20       0.7358        1.0000                 ?              0.3774
    0.75     29            30       0.002          1.0000                 ?              0.0018
    0.8       29            30       0.0105        1.0000                 ?              0.0093
    0.85     29            30       0.048          1.0000                 ?              0.0404
    0.9       29            30       0.1837        1.0000                 ?              0.1413
    0.95     29            30       0.5535        1.0000                 ?              0.3389
    0.75     28            30       0.002          1.0000                 ?              0.0086
    0.8       28            30       0.0442        1.0000                 ?              0.0337
    0.85     28            30       0.1514        1.0000                 ?              0.1034
    0.9       28            30       0.4114        1.0000                 ?              0.2277
    0.95     28            30       0.8122        1.0000                 ?              0.2586
    I couldn’t get CIs and hence …
    Second problem on which I would like your help is posted here:
     https://www.isixsigma.com/forum/showmessage.asp?messageID=41435
    Thanks in advance to those who would clear this confusion.
    Ssbb
     

    0
    #96557

    Ronald
    Participant

    Here is a website that focuses on Kappa and has a number of resources regarding the pros and cons of using such a metric for the evaluations of agreements.
    http://ourworld.compuserve.com/homepages/jsuebersax/kappa.htm

    0
    #96567

    Statman
    Member

    Hi SSBB,
    That should have been: BINOMDIST(1,20,0.25,TRUE) = 0.0243. 
    We are looking at the probability of 1 success out of 20 when the true proportion is 25%.  where a success is a mismatch.  The Confidence intervals were determined in Minitab using the 1-proportions test and exact calculations.
    Sorry for the error.  I had the right calculation and logic just the wrong formulas in the post. 
    Cheers,
    Statman

    0
    #106629

    winai
    Member

     To whom it may concern Please see MSA third edition page 132. Miss Rate of A,B and C = 5%,2% and 9% False Alarm Rate of A, B and C = 8%, 4% and 5% Could you please explain me, how to calculate it about miss rate and false alarm rate?

    0
Viewing 18 posts - 1 through 18 (of 18 total)

The forum ‘General’ is closed to new topics and replies.