Data is everything for a Six Sigma project. Before you can define a problem, analyze root causes, or confirm that improvements are taking hold, you need actionable data from a measurement system. Reliable numbers aren’t just a nice thing to have, but an expectation if you’re looking to have any effort take off. As such, a larger problem arises when we look at when your measuring system has an error. If your team is collecting data, charting, and making decisions with faulty data, you’re likely to see some larger, systematic faults.

This isn’t a hypothetical risk by any means. Measurement system errors are among the most common and underestimated pitfalls in process improvement work. When your measurement system has faults, the consequences ripple outward. Root causes might be misidentified, resources are wasted, and improvement efforts fall short. Since these errors are embedded in the data itself, it’s often unseen until someone thinks to look.

So, with that in mind, let’s look at how measurement systems fail, and how you can learn to trust your data again.

What Is a Measurement System?

A measurement system isn’t just a gauge or instrument. It takes in every element that goes into producing a data point. This means it’s looking at the likes of the operating taking the measurement, the equipment used, the methodology followed, the environment contributing to the measurement, and the attribute being measured. Each of the aforementioned components can introduce variation, which often gets mixed into data alongside the actual process variation you’re attempting to measure.

The goal of any Measurement System Analysis, or MSA, study is to quantify how much of your observed variation comes from the measurement system itself versus the process. If the ratio is too high, your data is telling you more about the flaws in your systems rather than the pertinent information you’re after.

Repeatibility and Reproducibility

Repeatability refers to the variation you see when an operator measures the same part multiple times with the same instrument under the same conditions. If the numbers vary, the instrument or procedure in use lacks overall consistency.

Reproducibility refers to the variation that appears when different operators measure the same part. If one operator is taking measurements and receiving dimensions of 10.2mm while another is receiving a measurement of 10.6mm, you’ve got a problem with reproducibility. It often indicates poorly defined procedures, inadequate training, or a measurement task that lacks objectivity.

Together, these elements contribute to gauge variation, or the noise floor, where your data can’t be trusted. For a common rule of thumb, you want to strive for gauge variation accounting for around 10% of your total observed variation. Any more, and that warrants investigation. If it’s exceeding 30%, you’ve got a serious problem on your hands.

What Happens When Your Numbers Are Consistently Wrong?

Bias is a systemic error, not random variation, with a consistent offset when looking at what your instruments are measuring and the true values. A scale might read a little heavier than it should, or calipers are lacking precision, often leaving things just a millimeter or two too thick. These introduce bias.

The tragic underpinning behind bias is that it often goes undetected for years. If you have entire departments making use of the same biased instruments, the numbers are internally consistent. There’s no reason to question the results, at least on the surface. Problems arise when you’re comparing measurements to reference standards, benchmarks from sister campuses, or bringing in a properly calibrated instrument. Suddenly, all of your processes look completely different.

Bias is corrected through calibration, but that’s only useful when done against a verified reference and done at regular intervals. As such, it’s vital to have a vetted, functioning calibration program in place.

Linearity and Bias

Linearity and bias are related, but the former has more nuance than the latter. Rather than having a consistent offset across measurements, linearity means that the bias changes depending on where you are in the measurement range. A gauge might be accurate at midrange values, but it drifts at the extremes of lower and upper values.

This has a profound impact on your process. Meaningful variation spread across a wide range takes on a critical dimension if you’re looking at your specification limits. A gauge’s overall precision only matters if it applies to the ranges you need. Without it, your gathered data points are going to give false confidence in the wrong places.

Linearity analysis takes measurements along the full operational range of a given instrument and compares them to a proven reference. If you’ve yet to do this for your critical measurement systems, you might have a glaring blind spot that needs addressing.

Stability

Stability refers to whether your measurement systems produce consistent results over time. Equipment wears out. Environmental conditions can change in subtle ways. Operators come and go. Any of these can cause a measurement system to drift, which isn’t a sudden jump by any means. Instead, it’s a slow, gradual shift that you might miss.

Unstable measurement systems yield their fair share of problems. Control charts might trigger false alarms or fail to detect real shifts in the process. You might attribute variation to a process when it actually occurs in the measurement system itself. This can cause the whole of a DMAIC project to run aground, ruining your efforts despite the best intentions.

Monitoring stability requires vigilance and regular evaluation of your measurement system. This isn’t a one-off evaluation at the start of a project. Measurement systems might have been adequate six months ago, but that isn’t necessarily going to apply to the ones you’re using today.

Attribute Agreement Analysis

Not all measurement systems are based on numerical values. Many industries make use of attribute data, with things like visual inspections, pass/fail conditions, or good/bad judgments used. These are equally prone to drift, but end up being much harder to quantify as a result.

Attribute Agreement Analysis is the MSA equivalent for these systems. It looks for agreement between differing operators, whether those attributes hold across repeated evaluations, and whether they apply to a known, documented standard.

Disagreement rates can be shocking when first measured. Attribute-based systems can often lead to operators being rather sure in their judgments, which leads to disagreements when compared to their colleagues. This occurs at rates that would be unacceptable in a numerical system. Without a proven system, your organization is going to have a measurement system that isn’t much better than just working off random chance.

Can You Detect Variation?

Resolution, or discrimination, refers to the smallest increment your measurement system can detect. If you’re trying to control a process that goes down to ten-thousandths of an inch, but your gauges are only going to hundredths, you’re flying blind. There will be repeated values, false patterns, and no way to detect any real shifts.

Your instrument resolution should measure at least one-tenth of the total process variation, ideally going much finer. Measurements that are too coarse result in something called chunky data. This means that your measurement system artificially limits the apparent variation, resulting in misleading patterns seen throughout your charts.

Conclusion

Every Six Sigma practitioner understands how easy it is to treat data as a given. You’re beginning projects with what historical data is present and going from there. The discipline and documentation of any measurement system is predicated on that assumption.

Before you invest time and energy into root cause analysis, regression models, or make false assumptions, ask how confident you are in the numbers your measurement systems are gathering.

If you lack any degree of confidence in your answer, the next step isn’t more analysis. Instead, it’s an MSA study. Your data might be telling a story, just not one that you’ll expect.