What Metric? What Test?

Andrew Kennett June 18, 2012Comments Off

This topic has 4 voices, contains 4 replies, and was last updated by Robert Butler 331 days ago.

Viewing 5 posts - 1 through 5 (of 5 total)

Author	Posts
Author	Posts
June 18, 2012 at 4:43 pm #183259
Kennett @AndrewKennett Reputation - 9 Rank - Aluminum	What metric? What test? One of our suppliers is wanting to change from approval based on a test results per batch (~100T batches) to blending batches and getting approval on calculated values. They will test each batch and then blend to meet our specs but in the long run won’t have time to test the blend. We suspect that the blending does not give a simple average or weighted average and there is a fair bit of measurement error (besides the huge sampling error). To test that their blending calculation gives a reasonable estimate of the test parameters we will, initially, get both their calculated value, their test results and we’ll do our own tests. So for each of 3 parameters we will have 3 data sets: calculated, supplier, inhouse. How best to test the accuracy calculated value?
June 19, 2012 at 7:24 am #183285
MBBinWI @MBBinWI Reputation - 2581 Rank - Titanium	Do you have experimental data on the blending process and how they control this process so that they know how much of batch A (at value X) to blend with batch B (at value Y) to get the acceptable value Q? If so, then you need to verify the measurement system for the blending operation to ensure that when they say they have added P tons you know that that is +/- some amount, and that that variation keeps the blend value at an acceptable level.
June 20, 2012 at 4:10 pm #183332
Mike Carnell @Mike-Carnell Reputation - 3168 Rank - Titanium	@AndrewKennett MBBinWI gave a great answer since I would venture to guess they are trying to reduce the amount of testing they do. This may be another approach. Have them control the process inputs and the process so that they don’t have to test to figure out if they did it right the first time. There was some methodology that was supposed to work on that idea. Just can’t quite think of the name of it right now. Six something………………? Just my opinion.
June 20, 2012 at 10:55 pm #183335
Kennett @AndrewKennett Reputation - 9 Rank - Aluminum	Thank you for your answers. we are running control charts so that should help visualise what is happening. And if there was only the calculated and 1 measured value for each parameter I could run a paired t-test but I have 2 labs testing. I suppsoe I could run a paired t-test calc v lab for each lab or I could run calc v average of the 2 labs but is there a test like the paired t-test for calc v lab 1 v lab 2?
June 21, 2012 at 5:24 am #183336
Robert Butler @rbutler Reputation - 2138 Rank - Silver	It sounds like “calc” is the gold standard and it sounds like there are two issues: 1. Relationship between calc and each of the labs. 2. Relationship between the labs. One possibility would be to run a paired t-test between calc and each of the labs. That would tell you if there is a difference between the gold standard and what the each of the labs are doing relative to the standard. Then, assuming you ran the test using split samples, you could run a paired t-test between the labs to see if they differ. Whether they differ or not you would probably want to also run a Bland-Altman test of agreement between the labs as an additional check on agreement and bias. Since you would be running multiple tests on the same sample populations you would need to adjust the p-value to account for this.

Viewing 5 posts - 1 through 5 (of 5 total)