iSixSigma

How to measure Audit results with a Sigma Value

Six Sigma – iSixSigma Forums Old Forums General How to measure Audit results with a Sigma Value

Viewing 12 posts - 1 through 12 (of 12 total)
  • Author
    Posts
  • #35528

    I’Anson
    Participant

     Has someone out there developed a methodology to measure their Audit results with a Sigma Value.
    We are a multi national aerospace company using standardized check sheets to perform our audit. We have thought off giving a value from 0 to 10 to each question but we feel that it is being to subjective for our auditors to do.
    We thought off giving a value to each MAJOR / MINOR corrective action raised but this was not conclusive.Any suggestions?

    0
    #100265

    Bob J
    Participant

    Daniel,
    You might be able to treat each check sheet item as an opportunity.  Defining an unchecked item as a defect you could then calculate a DPMO value which could be related to a Z value.
    Hope this helps…
    Bob J

    0
    #100268

    GAC
    Participant

    What is the value in assigning a sigma value? The purpose of an audit is to find compliance. Any finding otherwise is unacceptable.

    0
    #100274

    ASK
    Participant

    What are the units in this case or shud we take Units & Oppurtunites as the same.

    0
    #100278

    Bob J
    Participant

    ASK,
    If you mean units as measurements then yes, they will be the same as opportunities…
    Best Regards
    Bob J

    0
    #100378

    Daniel T.
    Participant

    GAC,
    The purpose is to be able to measure and compare our Audit results throughout our 8 plants located in 3 different country.

    0
    #100382

    GAC
    Participant

    Okay, just don’t forget to verify your measurement system. My experience is that audits are highly subjective.

    0
    #100389

    Thomas C. Trible
    Member

    Dan:
    You asked for suggestions about how you could “measure … audit results with a sigma value.”  Dan, I suggest that you not attempt to do this.  As others have replied to your question, the outcome, the results that you obtain, is highly dependent on the auditor. 
    Recall that all measurement outcomes are the result of a process.  Change the process, get a different outcome (number).  I submit to you that changing the auditor will induce so much variation in the process as to make such measurements problematic.  I suspect that the audit community would dispute my views, but I would like them to prove otherwise.
    If only audits were predictable in the sense of process predictability.  To use such audit data as a measurement tool, I believe that the burden of proof would have to be placed on the audit process to demonstrate that it is a stable process, and can be measured with the required uncertainty.  Can this be done?  I’d like to see this accomplished in practice.  Such an experiment would make an excellent paper for Quality Progress.
    TC Trible 

    0
    #100407

    Daniel T.
    Participant

    Thomas,
    What we have decided is to give a sigma value to only the deliverables audited.  For example, if item 1 of our check sheet is asking to audit 5 different duly completed form and each form has 17 fields to be entered, our DPMO would be 5×17=85 opportunities of defect.  Since it is a GO-NO GO (field entered or Not), the calculation becomes less subjective.
    We repeat the same methodology for item 2 …
    At the end of the audit, we can surely appreciated how the plant has meet the conformity of its deliverables.

    0
    #100436

    Thomas C. Trible
    Member

    Daniel T.
    Your reply demonstrates the importance of establishing operational definitions before any meaningful discussions can ensue.  In this case we both failed to establish an operational definition of “audit” that we could both agree on.
    In your example, you are using “counting,” not auditing, as in the traditional sense of quality auditing.  Quality auditing, if I may establish an operational definition here, involves data gathering, analysis, interpretation and drawing conclusions.  But quality auditing does not yield a quantitative measurement outcome in the sense of measuring product dimensions.  What one obtains with quality auditing is a qualitative sense of what systems or processes are effective, and those that are not.
    Counting does not require analysis, interpretations and drawing conclusions from data.  Those activities are major sources of variation in quality auditing.  That’s why auditing shouldn’t be used as a measurement outcome – too much uncontrolled variation.  On the other hand, counting does not require those attributes.  Apparently, we were talking about apples and oranges.
    TC Trible
     

    0
    #100438

    Mike Carnell
    Participant

    Daniel,
    Aside from the technical issues around using sigma as a response variable for you audit you need to check another aspect. In general it is not a sensitive measurement (the reason it is a poor metric for project work). It will take a large move to get any type of significant change in the sigma value. This can tend to demotivate the different groups. The lack of sensitivity can also make it difficult to distinguish between levels of performance (lack of resolution).
    GAC asked a legitimate question in where is the value. It may be subjective using a 1-10 score (you need to look up some information on Likert Scales) but disguising it as sigma doesn’t make it any less subjective.
    GAC was also partially correct in the audit intent as far as judging compliance. General audit theory is you check two things: 1. adequacy 2. compliance.
    I do not see where there is any value to the organization in the sigma rating. You will get more value in training auditors to be repeatable and reproducable.
    Good luck.

    0
    #106322

    cercech
    Participant

    TC Trible,
    Is it not possible after conducting a successful measurement system analysis of the audit team’s adherence to the operational definition of a defect (i.e. failed compliance) to then move forward with determining the appropriate sample size and then validate data independence for eventual compliance attribute testing?

     
     

    0
Viewing 12 posts - 1 through 12 (of 12 total)

The forum ‘General’ is closed to new topics and replies.