QA Calibration Variance

Six Sigma – iSixSigma Forums Operations Call Centers QA Calibration Variance

Viewing 5 posts - 1 through 5 (of 5 total)
  • Author
  • #250349


    How do we compute the variance per question and not the overall score?


    Facilitator     Evaluator1
    Score             Score
    Item1       10                   0
    Item2       0                 10
    Total        10                 10


    They seem calibrated as they have the same total but looking closely, but items 1 and 2 are both answered differently.

    Thank you


    Robert Butler

    It’s just the sums of the squares of the deviation from the mean divided by the total count minus 1.

    So, for a single item the average is  (10 + 0)/2 = 5

    The deviations from the mean are

    10 – 5 = 5

    0-5 = -5

    The squares of those deviations are

    5^2 = 25

    (-5)^2 = 25

    there are two observations so  2 – 1 = 1

    Therefore the variance per item is (25 + 25)/1 = 50  and the standard deviation is the square root of 50 = 7.07


    If this were a real situation, my first reaction would be: why bother with any calculations – better to find out why two different individuals are coming up with ratings that are the exact opposite of one another. As presented, if the two raters are the system, then there is no calibration whatsoever.


    Chris Seider

    @rbutler you are kind to help with homework. :)

    Stay safe and hope your family stays well!


    Robert Butler

    Things are fine on my end @cseider – hope all is well with you too.  As for the homework – it’s pretty quiet around these parts and I was feeling generous…    :-)



    This is off topic but I insist that we distinguish between QA and QC.  The main reason is that we can easily overlook QA when it’s conflated with QC.  Quality Control is testing, inspection, and analysis to verify that a product conforms to specifications, plus the related actions to handle nonconformance.  Quality Assurance is the systematic monitoring and evaluation of the various aspects of a project, service, or facility to ensure that standards of quality are being met.

Viewing 5 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic.