iSixSigma

Interpreting Two Survey Results

Six Sigma – iSixSigma Forums Old Forums General Interpreting Two Survey Results

Viewing 9 posts - 1 through 9 (of 9 total)
  • Author
    Posts
  • #52358

    Lee
    Participant

    In our facility a employee survey is routinely conducted, and this posing is about how to interpret the results.  Currently the Likert scale values of the responses are simply averaged, however the values varies from one survey to the next, as would be expected.
     
    I want to know what test is applicable for testing the significance of the changes of the various averages, i.e., Is 3.56 this year statistically different from last year’s 3.64?   Yes, I have looked in Implementing Six Sigma, and looked around in MiniTab and this site but did not find what seemed to be the ticket.  I’m wary of applying standard deviations and such, as the measurement is really human opinions rather than a defined process that is usually under the umbrella of the normal distribution.
     
    I do not have the option to change the questions, but the questions have remained the same over the last few years.  The sample size does vary, as does the portion by department, shift, and salary category.
     
    Thanks  — and enjoy your day

    0
    #184167

    Lee
    Participant

    Had forgotten about the Chi-squared test, seem like it is simple and applicable.

    0
    #184171

    MBBinWI
    Participant

    Eugene:  Averaging Likert scale is not appropriate.  If you are using 5 levels, with Very and Somewhat or Moderate being the upper and lower levels (with N/A or similar in the center) then you may want to add top 2 and bottom 2 together and track proportion changes over time.  One person’s Very may be another person’s Somewhat, so it is usually best practice to lump them together (unless you have very clear and distinct criteria on what constitutes each level).

    0
    #184172

    Lee
    Participant

    I agree that averaging is not appropriate, but that was what I found they are doing.  I give them credit for at least trying to make sense of the information.  I’m trying to bring a bit more sophistication to the data review and interpretation.
    I have started to challenge some concepts, like “If we can figure out how to get all 3s to be 4s fours…”.  To me that is non-sense because some people had mini-debates going on in their head about whether to mark 3 or 4, so even if you targeted tried to get all 3s to be 4’s, some 4’s would slip to 3.
    I intend to get the raw results and then by category and as a whole apply a chi-squared test to see if, at say a 80% confidence level, that the results are the same or not.  Does that seem reasonable to you?
    Thanks for the reply…

    0
    #184180

    Sloan
    Participant

    MBBinWI,
    This is a very timely topic for me. My business partners are concluding a satisfaction survey of their internal customers. They have always graded their surveys by taking a weighted average of responses to each question and coming up with a numeric summarized average value for each question like Eugene is showing (3.46 out of 5).
    My opinion is that if you want to know who is “satisfied” you simply count all of the responses in the categories that are satisfied (not neutral and not dissatisfied) and call that proportion of the survey population “satisfied.” It gives you a much more honest picture and it is easier to baseline and compare over time. I usually include three categories though, satisfied, neutral and dissatisfied. I also graph the distribution of responses for each question to see where the bulk of responses lie. You lose all of that if you take a average.
    Currently I’m caught in a debate about whether the scale should be 5 points or 7 points or 10 points. Last year it was 5 points and this year the want to change to 10. I warned them that they would lose their baseline if they did that and they said they would just take last year’s average based on the 5 point scale and double it to get their target for the 10 point scale. I cringed, I argued and I think I convinced them that it was not a good idea to do that.
    Does it always have to be this hard? Thanks for your insights on the subject.

    0
    #184181

    MBBinWI
    Participant

    Your instincts are good.  5 or 7 level scales are best (more options really doesn’t add any better fidelity, people generally bucket things in 5 categories – really like/dislike, kinda like/dislike, and neutral or doesn’t matter, sometimes it is good to add 2 more so that people can have 3 areas like, dislike, and shades of the middle).  10 would not provide a middle point.
    Since different people react to these surveys differently (some anchor on the max and subtract from there, others anchor on the middle and adjust from there, few anchor on the bottom) averages have little meaning and tracking an average after changing the scale would prove hazardous.
    If you’re going to change the format, why not go to one where they allocate points to the different items based on level of satisfaction?  Takes some technology to implement in a satisfactory manner, but that would provide a much better resolution (allocate 100 points to the items or such).

    0
    #184182

    Sloan
    Participant

    I’m afraid that mule is already out of the barn with a 10-point scale despite my advice to the contrary. At least I feel a little vidicated that someone else thinks I was right.
    I am being assigned to a different project so I won’t be the one trying to interpret the results and tie them back to last year’s number. I feel bad for the people that will have to do it. They will be pressed to compare last year with this year and someone will be put in a tight spot.

    0
    #184200

    Ivan
    Participant

    You can find an exellent guide to CSM on http://www.leadershipfactor.co.uk/other/1/index.phtml
     

    0
    #184207

    Sloan
    Participant

    Ivan,
    I disagree. That was not so much a guide as it was a commercial. And not a very informative commercial at that. Your link told me nothing.

    0
Viewing 9 posts - 1 through 9 (of 9 total)

The forum ‘General’ is closed to new topics and replies.