Repeatability and reproducibility are two ways that scientists and engineers measure the precision of their experiments and measuring tools. They are heavily used in chemistry and engineering and are typically reported as standard deviations in scientific write-ups. While these two measurements are both used in many types of experiments, they are quite different and offer valuable insights into the work being done.

What is Repeatability?

Simply put, repeatability is the variability of a measurement. For it to be established, the measurement must be taken by the same person, under the same conditions, using the same instruments, and in a short amount of time.

The Benefits of Repeatability

Repeatability is important for researchers to verify their results. If you can produce the same result in an experiment using the same setup and methods under the same exact conditions in a relatively short amount of time, you can assume the results are not simply by chance. Then you can work towards creating reproducibility.

Repeatability is also essential in determining the accuracy of your measuring tools. This could be repeatedly measuring the length of a bolt with the same caliper and recording the results. If the results are within an acceptable range, you can say the tool produces repeatable measurements.

How to Calculate Repeatability

In order to calculate repeatability, you need to perform a repeatability test. When collecting data, you must ensure that certain criteria are met. This includes ensuring the exact same:

1. Conditions
2. Operator
3. Equipment
4. Location
5. Item
6. Method

Once you have all six of these criteria met, you can move on to performing a repeatability test. First, you will need to choose what you are going to measure. Depending on your experiment, this could be voltage, length, pressure, temperature, or a number of other measurements. Then you will need to set a measurement range. This is typically a range of low to high values.

After you have determined your measurement range, the next thing to do is choose test points. For a linear experiment, this would be two points. Usually, this is 10% and 90% of the measurement range. Depending on the project and if available, calibration points should be chosen for the best results. If your experiment is non-linear, you should pick three or more points to prevent errors.

The only thing left to do is perform the experiment using the criteria and parameters listed above and collect your data. You will want to perform the test 20 to 30 times if practical. Some experiments are time-consuming, and it is only realistic to perform five or fewer tests. You should choose whatever is the most appropriate for your experiment.

Once you have all your data collected, it is time to analyze it. You should calculate the degrees of freedom, mean, and standard deviation. When dealing with multiple sets of data, you should use the method of pooled variance when making your calculations. Your standard deviation will be the value for repeatability.

What is Reproducibility?

Reproducibility measures the degree of agreement of an experiment or study performed by different individuals in different locations and with different instruments. An experiment that is reproducible if it can replicate the results anywhere and by anyone who properly follows the procedure.

The Benefits of Reproducibility

Achieving reproducibility allows for more thorough and accurate research. It allows teams to reduce errors and ambiguity before moving a project from the development phase to the production phase. Reproducibility benefits the researcher and the research community as a whole.

How to Create Reproducibility

Creating reproducibility is simply the process of determining if your experiment can be replicated by a different individual or team using the same setup and provide the same results. Typically, an experiment needs to be performed successfully multiple times by different teams before it can be considered reproducible.

Repeatability vs. Reproducibility: What’s the Difference?

Reproducibility allows for more accurate research, whereas repeatability measures that accuracy and confirms the results. Both are a means to evaluate the stability and reliability of an experiment and are key factors in uncertainty calculations of measurements.

Repeatability vs. Reproducibility: Who would use repeatability and/or reproducibility?

Both repeatability and reproducibility are essential in nearly all areas of science, including chemistry, computer science, data science, and engineering. A line operator in a manufacturing facility may rely more on repeatability, whereas a chemist developing a new chemical reaction would strive to achieve reproducibility of their experimental method and rely on repeatability data to determine its accuracy.

Choosing Between Repeatability and Reproducibility: Real World Scenarios

Let’s say a quality inspector is measuring the weight of different sized bolts to ensure they are within the acceptable range before sending them through to the next step in the manufacturing process. To determine the reliability of the scale, the head engineer wants the operator to weigh each bolt multiple times and record their results. Essentially, the operator is performing a repeatability test. The engineer notices that the scale appears to be having a significant variation in bolts that weigh more than 50 grams. The conclusion is that a different scale capable of producing repeatable measurements on the same bolts over 50 grams needs to be installed at the quality inspector’s station. This simple repeatability test allowed the engineer to catch a potential area of inaccuracy in the manufacturing process and potentially avoid costly quality and functional issues in the end product.

Reproducibility is a major goal of research teams and collaborations in the field of chemistry. Let’s say a team of chemists has produced a chemical reaction that can create a nickel complex that absorbs near-infrared light. The team has designated one of its members to perform the test multiple times and record their data. They have come to the conclusion that their experiment is repeatable. The team is in collaboration with another team working on the same goal of producing safer materials for solar panels. They send their method to the other team and ask them to perform the experiment multiple times and record their data. Several of the other teams’ members reproduce the same results as the original team. The experiment can now be deemed reproducible, and the team can move on to improving the process so it can be scaled for production.

Summary

Depending on what industry you work in, you may need to use repeatability and/or reproducibility tests. If you are simply trying to determine the accuracy or consistency of a measuring tool, repeatability tests are likely all you need. However, if you are developing new processes and/or products, both repeatability and reproducibility are essential in the process of moving your ideas from planning to experimentation to production.