% Tolerance vs Repeatability

Six Sigma – iSixSigma Forums General Forums Methodology % Tolerance vs Repeatability

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
  • #235499


    Hello! I had a quick question about a Gage R&R study I recently did measuring the Percentage of Vanadium in an alloy we make.

    I did the usual 10 parts, 3 operators, 2 measurements.

    I’m getting a high %Study Var due to repeatability (38.70%). Which would indicate that the Gage is not acceptable for use, but a %Tolerance of 3.75%.

    Is this a situation where the %Tolerance is a more useful number than Total Gage R&R? From what I can see these are great results (Our specification for % Vanadium is 7±0.5%)

    If it helps at all each Part is a sample from a separate lot, so I don’t realistically expect them to have exactly the same % composition

    Here is my Minitab Data

    Gage R&R Study – ANOVA Method

    Two-Way ANOVA Table With Interaction

    Source DF SS MS F P
    Sample 9 0.0123235 0.0013693 27.4659 0.000
    Tech 2 0.0000346 0.0000173 0.3473 0.711
    Sample * Tech 18 0.0008974 0.0000499 1.5300 0.147
    Repeatability 30 0.0009775 0.0000326
    Total 59 0.0142330

    α to remove interaction term = 0.05

    Two-Way ANOVA Table Without Interaction

    Source DF SS MS F P
    Sample 9 0.0123235 0.0013693 35.0560 0.000
    Tech 2 0.0000346 0.0000173 0.4433 0.644
    Repeatability 48 0.0018749 0.0000391
    Total 59 0.0142330

    Gage R&R
    Source VarComp (of VarComp)
    Total Gage R&R 0.0000391 14.98
    Repeatability 0.0000391 14.98
    Reproducibility 0.0000000 0.00
    Tech 0.0000000 0.00
    Part-To-Part 0.0002217 85.02
    Total Variation 0.0002608 100.00

    Process tolerance = 1

    Study Var %Study Var %Tolerance
    Source StdDev (SD) (6 × SD) (%SV) (SV/Toler)
    Total Gage R&R 0.0062498 0.0374987 38.70 3.75
    Repeatability 0.0062498 0.0374987 38.70 3.75
    Reproducibility 0.0000000 0.0000000 0.00 0.00
    Tech 0.0000000 0.0000000 0.00 0.00
    Part-To-Part 0.0148897 0.0893381 92.21 8.93
    Total Variation 0.0161481 0.0968888 100.00 9.69

    Number of Distinct Categories = 3

    • This topic was modified 3 years, 10 months ago by Lkasprz.
    • This topic was modified 3 years, 10 months ago by Michael Cyger.

    Mike Carnell

    Look at you Number of Distinct Categories. You have 3. That means your gage has no resolution. This is why the first number you check on a R&R is Distinct Categories. When this number is low (they say 4 I prefer 5-6) none of the rest of the numbers mean anything.

    I have never been a big fan of % Study Variation either. % Tolerance is what we have to deal with in the real world.



    Thank you for your response!


    Mike Carnell

    You are welcome. If your samples are all the same (or extremely close) it can cause this to happen.

    You said this was vanadium. There can be a couple issues. Depending on where in the process the samples come from. If it has been through some refining it is probably very homogeneous which will mean it is basically seeing the same sample repeatedly. The calculations will assume it just cannot tell the difference. If it has not been through very much refining it may have a lot of contaminates. Your repeatability (is typically an operator issue) will be poor unless you make sure you are reading the exact same spot every time.

    When you are doing things like ore you can walk into a lot of things that have nothing to do with the measurement device. Sample prep and technique are also part of the measurement system and affect the result. These are suggestions but if you get into repeatability look at the operator controllable parts of the operation. If it is reproducibility check the measurement device.

    One you mark a specific spot on something to measure put it under a microscope so you understand the surface you are measuring. If you see something like flakes then you know your surface will be changing. You need consistency in the sample to understand the gage.



    Yes, Our results are all very close together

    101324715D 6.854 6.853 6.853 6.847 6.870 6.843
    101326238 6.882 6.885 6.878 6.871 6.891 6.901
    101326240 6.863 6.857 6.861 6.872 6.863 6.863
    101326239 6.866 6.866 6.868 6.868 6.876 6.859
    101326241 6.842 6.835 6.843 6.845 6.843 6.836
    101326262 6.855 6.849 6.850 6.843 6.847 6.843
    101326237 6.842 6.837 6.843 6.847 6.844 6.846
    101324715A 6.849 6.855 6.860 6.847 6.850 6.842
    101321247 6.862 6.858 6.847 6.846 6.854 6.855
    101329074 6.879 6.883 6.882 6.879 6.878 6.883

    I could see there being a little bit of variation as to how the sample is oriented when placed into the instrument, could doing something like making sure the samples are locked in the same position for each operator improve the repeatability number?

    Thanks again!

    • This reply was modified 3 years, 10 months ago by Lkasprz.
    • This reply was modified 3 years, 10 months ago by Lkasprz.
    • This reply was modified 3 years, 10 months ago by Michael Cyger.

    Mike Carnell

    Those values are really tight. I am guessing it thinks the gage just can’t see the difference. What does your spec look like?

    I prefer to fixture for a R&R study just because I want to be able to tell how much of an issue the gage is and how much of an issue the sample is.


    Chris Seider

    Have you done a capability analysis with the present measurement system? You should try to sample across your known sources of variation to help get a more process representative sample to do a gage R&R.

    is spot on as usual. If your samples for the gage R&R don’t represent your process well, you’ll get confusing results.

    If you haven’t gathered samples with representative values across your typical process spectrum of results, you run the risk of not knowing how the gage performs. It’s possible it can’t read well “high” or “low” readings and you wouldn’t find the deficiency if you sample across a small part of your capability.


    Chuck White

    Good discussion, but I have to disagree with @Mike-Carnell on a couple of points. The first is that % Tolerance is a better measure than % Study Variation. They tell you two completely different things, and which one is more relevant depends on the reason for doing the gage study.

    % Tolerance tells you how well you can distinguish good parts from bad parts. If you are doing the study to qualify a new gage for production, then I agree with Mike — this is the better metric.

    % Study Variation tells you how well you can “see” other sources of variation through the measurement error. If you are doing the study to qualify a gage for a Six Sigma project (or any kind of variation reduction project), this in my opinion, is a much better metric. If you can’t see the variation sources because there is too much noise in your measurement system, you’ll never be able to identify your KPIVs even if you have a good % Tolerance result.

    The second point is that a low Number of Distinct Categories always means you do not have enough resolution. That could be the case, but Distinct Categories actually means how many categories can you consistently separate your measurement data into, taking the measurement error into account. In other words, how many measurement error distributions fit into your overall distribution. If there is a lot of measurement noise, you can have great resolution but still have poor Distinct Categories. Think of a micrometer with a loose anvil — your resolution is 0.001 mm, but you may only be able to consistently distinguish parts that are more than 0.02 mm apart.

    By the way, % Study Variation is perfectly negatively correlated with Number of Distinct Categories — if you have a high % Study Variation, you always have a low Number of Distinct Categories. If you look at the formulas for each, you can see how one can be calculated from the other (let me know if you want me to elaborate).

    So to the original question from @Lkasprz, either the samples are not representative of the entire process, or the process is very capable. If it is the later case, I would say you likely have no need for a variation reduction project, and are just interested in whether you can distinguish good batches from bad batches. In that case your measurement system appears to be adequate. If it is the former case, then you need to go back a get a more representative sample for your gage study. Or, if you are using Minitab, you can click the Options button when running the analysis, and enter a historical process standard deviation if you have that information.



    @jazzchuck, thank you for your input!

    This is definitely more of a “telling good parts vs bad parts” kind of situation. We are just checking the sample against the spec (6.70% – 7.30% Vanadium), and passing/failing it.

    I randomly selected samples that were produced over a time span of a few months (they were just hanging out in a box, waiting to be scrapped), so my hope is that they are pretty representative of what we are producing.

    Thanks again everyone! This has been very helpful.


    Mike Carnell

    Lkasprz The thing that prompts most R&R studies is exactly what you said “can I make the measurement.” If you can’t that costs you actual money in the real world right now. You also may want to understand guard banding from the % tolerance stand point. Whatever the % tolerance is there is a band that wide around the USL and LSL where a measurement in that band can actually be either way. Those can be expensive.

    If you want to play with visibility of other sources of variation that is probably better left to when you know if you are shipping good or bad product. At the very least your customer appreciates it.

    Just my opinion.


    Chris Seider

    suggestion: If possible, change the measurement system to keep the vanadium measurements and not the good/bad attribute measurements in your database.

    You’ll learn lots in the future if you can.

Viewing 11 posts - 1 through 11 (of 11 total)

You must be logged in to reply to this topic.