iSixSigma

Capability Analysis for ordinal data

Six Sigma – iSixSigma Forums Old Forums General Capability Analysis for ordinal data

Viewing 14 posts - 1 through 14 (of 14 total)
  • Author
    Posts
  • #48451

    Six Sigma guy
    Member

    Any thoughts on how to do capability analysis for ordinal/nominal data. Say for example we do CSAT Surveys and we get responses based on 1-5 ratings on 10 parameters. How do we do capability analysis for this?
    1.Do we consider average rating for each parameter and do capability analysis with some spec limit?
    2. Do we consider top two boxs and do capability analysis with some spec limit?
    3. Do we consider anything other than 4,5 to be defects and do capability analysis?
    4. Or something else..
    Capability analysis for ordinal/nominal data wasnt taught to us in our BB class. Do they teach it in MBB or is it training company based?In fact nominal/ordinal regression itself wasnt taught to us….
    Thanks and Regards
    Six Sigma guy
     

    0
    #163385

    DoNotKnowSixSigma
    Participant

    Capability Analysis is only done for continuous and attribute (defects and defectives) and not on a rating scale like Customer Survey.
    Better do the ordinal/nominal regression and determine the scales you want to cover plus the frequency.
     

    0
    #163390

    Six Sigma guy
    Member

    Any specific reason why it shouldnt be done or cant be done?

    0
    #163394

    Tony Bo
    Member

    You dont do Capability  on surveys….you would do it on the process you are surveying…..maybe do paretos on the survey data to find out where or what to improve…..
    Does that help…?

    0
    #163455

    Six Sigma guy
    Member

    Thanks Tony for your reply. I asked this question out of curiosity. I know what I should be doing for surveys was wondering if capability analysis could be done… Dont take surveys alone as an example it could be any nominal or ordinal data..
    If you consider CSAT as a metric then?
    I was telling myself that this question could generate varied response but not much has come- Maybe my question is stupid? Or havent people thought about this possibility?
    Thanks

    0
    #163464

    Ken Feldman
    Participant

    Of course you can do capability on an ordinal scale such as a survey.  Find it in Minitab:
    Stat > Quality Tools > Attribute Agreement Analysis
    Use attribute agreement analysis to assess the agreement of nominal or ordinal ratings given by multiple appraisers. The measurements are subjective ratings by people rather than direct physical measurements. Examples include:
    ·    Automobile performance ratings
    ·    Classification of fabric quality as “good” or “bad”
    ·    Ratings of wine color, aroma, and taste on a 1 to 10 scale
    In these situations, quality characteristics are difficult to define and assess. To obtain meaningful classifications, more than one appraiser should classify the response measure. If the appraisers agree, the possibility exists that the ratings are accurate. If the appraisers disagree, rating usefulness is limited.
    As an output you can also get Kendall’s coefficient which is used for ordinal data and measures the degree to which people miss.  Check it all out.

    0
    #163479

    Tony Bo
    Member

    Consider Darths response…he makes very good points. You can do a capability analysis on anything.  But to do it on surveys…basically doing it on a subjective response…that to me is questionable.  Let me ask…lets just say you do a capability analysis on survey results and get less than desired results….what do you plan to do next…?

    0
    #163483

    Ken Feldman
    Participant

    Tony,
    We used this in a situation where we were having managers evaluate their people on a subjective evaluation scale that was used to judge a portion of the compensation. We did a MSA on the ordinal data which in essence was a capability of the measurement system (managers) to accurately and consistently judge people. Certainly a recipe for disaster. We found out that the system was not capable of adequately evaluating the people. We did a lot of work to improve the repeatability and reproducibility through scenario writing and calibrating the managers as to what each number on the scale means. While not a classic capability study yielding Cpk it was a way of assessing the capability of the measurement system.

    0
    #163486

    Tony Bo
    Member

    Thanks Darth..!! Totally agree and see/understand how you applied it.  I apologize if it seemed like my post was addressed to you or questioning your method….it was really toward Six Sigma guy to his original post…it just sounded like he wanted to do a straight up capability analysis on survey results…which to me, can be done, …but is it the right thing to do. 
     
     

    0
    #163547

    Six Sigma guy
    Member

    Thanks Darth/Tony. My question was just for knowledge purposes and not to do any capability as such in reality. I was asked this question by a senior person so i was wondering what should be the correct response to that. Attribute agreement analysis helps for MSA but i was wondering if we could generate any Z or CP/CPK for this?
    Consider this scenario 
    In a call center we have end user CSAT survey after every call. A set of questions is asked say 10 questions and customer needs to rate from 1-5 for each parameter and target is 4.5 and minimum 3 needs to be scored. A list of scores are generated. Now how do we know the capability?
    Why shouldnt it be done? is it because its subjective?
    Thanks and Regards,
    SSG

    0
    #163579

    Tony Bo
    Member

    Hi SSG..
    In your scenario..you can do the capability, you would have to do it using the discrete method, because of your scoring.  In my opinion (again, its my opinion)…if I were giving advice to my BB….i would not do it this way.  I would actually take the survey, pareto out reasons why we are getting bad survey results, attempt to get from the surveys any “customer specs”…and then apply them to processes they are unhappy with…and possibly wrap a project around it to improve.  Improve the processes will in turn improve your survey results.  
    Just my input… 

    0
    #163691

    Six Sigma guy
    Member

    Thanks Tony. Appreciate your help. Another reason probably what I was looking at is to do current baseline and improve on them which we can do by pareto,root cause etc..
    Regards,
    SSG

    0
    #164252

    Fontanilla
    Participant

    Just to help (maybe!) clarify why you won’t do capability analysis with this study, you are probably making a huge assumption that the “distance” between levels of your rating scale are the same.  Most likely, this is not true.  Most likely they are non-linear.  That is, the difference between say a “1” and a “3” in the responses is mathematically a “2” which Minitab or other software will dutifully calculate for you.  However, did it really take 2 “units” of human, emotional response to go from a 1 to a 3?  Now ask that of all the respondents.  You get a highly non-linear set of responses.  To use a linear method of analyzing it just doesn’t make practical sense. 
    What makes practical sense is the Pareto approach mentioned earlier.  Put on your “Graphical” cap (from your Practical, Graphical, Analytical wardrobe), and then your “Practical” hat.  Forget about the “Analytical” at this point.  You may come back to it later, like Darth did, after much work.
    Regards

    0
    #175005

    Hunter1897
    Participant

    Actually, the tests you mention in Minitab are not Process Capability tests for ordinal data, but a revised version of the Attribute MSA that was in previous versions.  The tools are used to tell us if we can rely on the measurement system being used (such as in a survey where you might have ratings such as Bad, Good, Great, or something like that). 
    For ordinal data, the best thing to do is to make a determination as to what level in the scale is acceptable to you and your customers, and strive to make improvements in the areas where you are below either on average or on median.  I like to use the median value instead of averages for survey type data.

    0
Viewing 14 posts - 1 through 14 (of 14 total)

The forum ‘General’ is closed to new topics and replies.