iSixSigma

How good is my in-house test method

Six Sigma – iSixSigma Forums General Forums Methodology How good is my in-house test method

Viewing 6 posts - 1 through 6 (of 6 total)
  • Author
    Posts
  • #53678

    roarty
    Participant

    I have two test methods for measuring abrasion – ‘Sliding Abrasion’ and ‘Impact Abrasion’. I wish to measure the degree of sliding abrasion of twenty test samples then measure the twenty test samples for impact abrasion. Sliding abrasion is an industry standard ASTM test method; impact abrasion was developed in-house and to my knowledge has never been statistically validated.

    Now that I have compiled all my results it is clear that the results for sliding abrasion are well spread (high std. dev.) and I can easily differentiate between good and bad samples. For impact abrasion the results are very similar to each other, there is little difference between samples and I am not confident that I can say one sample is better than another based on such small differences. It is my suspicion the impact abrasion test is too severe and abrades all samples equally regardless of the quality of the sample.

    My question is how do I prove or disprove my suspicion. At the moment I have drawn a bar chart for each test method and it provides a good visual representation of the spread of values for each test. I have also calculated the std. dev. for each test and expressed it as a percentage of the average value. So I have demonstrated the spread of values but how do I demonstrate that the test with the low std. dev. fails to differentiate between samples.

    0
    #191070

    MBBinWI
    Participant

    You need more decimal places for your measurement device. The question should be, do you need that level of precision for the requirements that you are trying to satisfy?

    0
    #191071

    roarty
    Participant

    Thank you for your reply MBBinWI,

    You are right; I do not need a great level of precision.

    Increasing the level of precision would be useful if I was seeing no differences between samples but I am seeing differences but I’m not sure if the differences I observe are purely experimental error or are they real (albeit small) differences.
    Each measurement of impact abrasion is an average of five readings. So I have in effect, 20 groups of 5 readings. One approach may be to use ANOVA to determine if the variance within each group (experimental error) is greater than the variance between groups (real performance difference).

    Does this sound like a sound approach? A concern is; let’s say I use ANOVA to establish that there are differences between samples. Does this mean that the test method is distinguishing between all twenty samples or just a few of the samples out of the twenty. My fear is that I draw the wrong conclusion from ANOVA based on the several samples which may be distinguishable from the rest but the majority of samples are much the same as each other.

    0
    #191080

    Robert Butler
    Participant

    You said “Each measurement of impact abrasion is an average of five readings. So I have in effect, 20 groups of 5 readings.” Question: How were these five readings obtained? That is, did you take a sample, abrade it once, take a measurement, and then find a new sample, abrade it once and take another measurement or did you just take a single sample, abrade it 5 times, record the results after each abrasion and then take the average of the 5 results?

    If it was the latter then you are violating one of the key requirements of ANOVA – sample independence. What you have in the second case is repeated measures and you will have to account for subject effect before you can make any statements about your process differences. The good news is that in this instance all that you need to do is account for subject in addition to treatment – the way you do this is set up a two way ANOVA with the groups being samples and treatments – run the ANOVA on treatments and samples and see what you get.

    If your measurements were indeed repeat measures you will probably need a reference to justify your approach to the analysis. The best description I know of can be found on pp. 254-259 of Regression Methods in Biostatistics by Vittinghoff, Glidden, Shiboski, and McCulloch. In your case, abrasion method will replace pill type.

    0
    #191081

    roarty
    Participant

    Thank you Robert,

    You are correct in assuming that I have abraded the same sample five times and you have raised an issue that had crossed my mind but to be honest I had not given it serious consideration. Clearly the first treatment is abrasion to a freshly prepared sample while all subsequent treatments are to a previously abraded sample.

    Thanks to your help, I now feel confident that I am taking the right approach and I should be able to validate this test method.

    0
    #191082

    Robert Butler
    Participant

    Ok, since we are talking repeat measures then there is one additional thing you will need to do – use the raw data – do not compute an average. Also since it is a repeat measures problem you will definitely want to read the reference I mentioned to make sure you are giving your computer program the right instructions with respect to the analysis.

    0
Viewing 6 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.