Capability Analysis for ordinal data
Six Sigma – iSixSigma › Forums › Old Forums › General › Capability Analysis for ordinal data
- This topic has 13 replies, 6 voices, and was last updated 14 years ago by
Hunter1897.
-
AuthorPosts
-
October 19, 2007 at 4:00 am #48451
Six Sigma guyMember@Six-Sigma-guyInclude @Six-Sigma-guy in your post and this person will
be notified via email.Any thoughts on how to do capability analysis for ordinal/nominal data. Say for example we do CSAT Surveys and we get responses based on 1-5 ratings on 10 parameters. How do we do capability analysis for this?
1.Do we consider average rating for each parameter and do capability analysis with some spec limit?
2. Do we consider top two boxs and do capability analysis with some spec limit?
3. Do we consider anything other than 4,5 to be defects and do capability analysis?
4. Or something else..
Capability analysis for ordinal/nominal data wasnt taught to us in our BB class. Do they teach it in MBB or is it training company based?In fact nominal/ordinal regression itself wasnt taught to us….
Thanks and Regards
Six Sigma guy
0October 19, 2007 at 12:16 pm #163385
DoNotKnowSixSigmaParticipant@DoNotKnowSixSigmaInclude @DoNotKnowSixSigma in your post and this person will
be notified via email.Capability Analysis is only done for continuous and attribute (defects and defectives) and not on a rating scale like Customer Survey.
Better do the ordinal/nominal regression and determine the scales you want to cover plus the frequency.
0October 19, 2007 at 3:22 pm #163390
Six Sigma guyMember@Six-Sigma-guyInclude @Six-Sigma-guy in your post and this person will
be notified via email.Any specific reason why it shouldnt be done or cant be done?
0October 19, 2007 at 4:15 pm #163394You dont do Capability on surveys….you would do it on the process you are surveying…..maybe do paretos on the survey data to find out where or what to improve…..
Does that help…?0October 22, 2007 at 6:39 am #163455
Six Sigma guyMember@Six-Sigma-guyInclude @Six-Sigma-guy in your post and this person will
be notified via email.Thanks Tony for your reply. I asked this question out of curiosity. I know what I should be doing for surveys was wondering if capability analysis could be done… Dont take surveys alone as an example it could be any nominal or ordinal data..
If you consider CSAT as a metric then?
I was telling myself that this question could generate varied response but not much has come- Maybe my question is stupid? Or havent people thought about this possibility?
Thanks0October 22, 2007 at 12:50 pm #163464
Ken FeldmanParticipant@DarthInclude @Darth in your post and this person will
be notified via email.Of course you can do capability on an ordinal scale such as a survey. Find it in Minitab:
Stat > Quality Tools > Attribute Agreement Analysis
Use attribute agreement analysis to assess the agreement of nominal or ordinal ratings given by multiple appraisers. The measurements are subjective ratings by people rather than direct physical measurements. Examples include:
· Automobile performance ratings
· Classification of fabric quality as “good” or “bad”
· Ratings of wine color, aroma, and taste on a 1 to 10 scale
In these situations, quality characteristics are difficult to define and assess. To obtain meaningful classifications, more than one appraiser should classify the response measure. If the appraisers agree, the possibility exists that the ratings are accurate. If the appraisers disagree, rating usefulness is limited.
As an output you can also get Kendall’s coefficient which is used for ordinal data and measures the degree to which people miss. Check it all out.0October 22, 2007 at 3:34 pm #163479Consider Darths response…he makes very good points. You can do a capability analysis on anything. But to do it on surveys…basically doing it on a subjective response…that to me is questionable. Let me ask…lets just say you do a capability analysis on survey results and get less than desired results….what do you plan to do next…?
0October 22, 2007 at 3:56 pm #163483
Ken FeldmanParticipant@DarthInclude @Darth in your post and this person will
be notified via email.Tony,
We used this in a situation where we were having managers evaluate their people on a subjective evaluation scale that was used to judge a portion of the compensation. We did a MSA on the ordinal data which in essence was a capability of the measurement system (managers) to accurately and consistently judge people. Certainly a recipe for disaster. We found out that the system was not capable of adequately evaluating the people. We did a lot of work to improve the repeatability and reproducibility through scenario writing and calibrating the managers as to what each number on the scale means. While not a classic capability study yielding Cpk it was a way of assessing the capability of the measurement system.0October 22, 2007 at 4:01 pm #163486Thanks Darth..!! Totally agree and see/understand how you applied it. I apologize if it seemed like my post was addressed to you or questioning your method….it was really toward Six Sigma guy to his original post…it just sounded like he wanted to do a straight up capability analysis on survey results…which to me, can be done, …but is it the right thing to do.
0October 23, 2007 at 6:04 am #163547
Six Sigma guyMember@Six-Sigma-guyInclude @Six-Sigma-guy in your post and this person will
be notified via email.Thanks Darth/Tony. My question was just for knowledge purposes and not to do any capability as such in reality. I was asked this question by a senior person so i was wondering what should be the correct response to that. Attribute agreement analysis helps for MSA but i was wondering if we could generate any Z or CP/CPK for this?
Consider this scenario
In a call center we have end user CSAT survey after every call. A set of questions is asked say 10 questions and customer needs to rate from 1-5 for each parameter and target is 4.5 and minimum 3 needs to be scored. A list of scores are generated. Now how do we know the capability?
Why shouldnt it be done? is it because its subjective?
Thanks and Regards,
SSG0October 23, 2007 at 3:29 pm #163579Hi SSG..
In your scenario..you can do the capability, you would have to do it using the discrete method, because of your scoring. In my opinion (again, its my opinion)…if I were giving advice to my BB….i would not do it this way. I would actually take the survey, pareto out reasons why we are getting bad survey results, attempt to get from the surveys any “customer specs”…and then apply them to processes they are unhappy with…and possibly wrap a project around it to improve. Improve the processes will in turn improve your survey results.
Just my input…0October 24, 2007 at 4:31 am #163691
Six Sigma guyMember@Six-Sigma-guyInclude @Six-Sigma-guy in your post and this person will
be notified via email.Thanks Tony. Appreciate your help. Another reason probably what I was looking at is to do current baseline and improve on them which we can do by pareto,root cause etc..
Regards,
SSG0November 1, 2007 at 6:35 pm #164252Just to help (maybe!) clarify why you won’t do capability analysis with this study, you are probably making a huge assumption that the “distance” between levels of your rating scale are the same. Most likely, this is not true. Most likely they are non-linear. That is, the difference between say a “1” and a “3” in the responses is mathematically a “2” which Minitab or other software will dutifully calculate for you. However, did it really take 2 “units” of human, emotional response to go from a 1 to a 3? Now ask that of all the respondents. You get a highly non-linear set of responses. To use a linear method of analyzing it just doesn’t make practical sense.
What makes practical sense is the Pareto approach mentioned earlier. Put on your “Graphical” cap (from your Practical, Graphical, Analytical wardrobe), and then your “Practical” hat. Forget about the “Analytical” at this point. You may come back to it later, like Darth did, after much work.
Regards0August 19, 2008 at 11:59 am #175005
Hunter1897Participant@Hunter1897Include @Hunter1897 in your post and this person will
be notified via email.Actually, the tests you mention in Minitab are not Process Capability tests for ordinal data, but a revised version of the Attribute MSA that was in previous versions. The tools are used to tell us if we can rely on the measurement system being used (such as in a survey where you might have ratings such as Bad, Good, Great, or something like that).
For ordinal data, the best thing to do is to make a determination as to what level in the scale is acceptable to you and your customers, and strive to make improvements in the areas where you are below either on average or on median. I like to use the median value instead of averages for survey type data.0 -
AuthorPosts
The forum ‘General’ is closed to new topics and replies.