WEDNESDAY, JULY 30, 2014
Font Size
Topic Surveys – How to determine representative sample

Surveys – How to determine representative sample

Home Forums General Forums General Surveys – How to determine representative sample

This topic contains 7 replies, has 3 voices, and was last updated by Avatar of Robert Butler Butler 2 years, 2 months ago.

Viewing 8 posts - 1 through 8 (of 8 total)
  • Author
    Posts
  • #169007 Reply

    I am doing a VOC survey regarding some of my division’s products and services. Can anybody tell me how to determine the number of responses I would need to get back on a population of 1,000 in order to be confident that I have a “representative” response?

    #169009 Reply
    Avatar of Robert Butler
    Butler
    Member
    Reputation - 0
    Rank - Aluminum

    The answer to your question is that, as written, there isn’t one.

    Meaningful questions about sample size relevance require knowledge of the population to be sampled and the ability to elicit responses from designated representatives of that population. If you have detailed demographic information about the population, are able to use that demographic data to identify meaningful strata within that population, and have the means to not only identify random individuals within those strata but to elicit responses from those specified individuals, then you are in a position to as questions about sample size relevance. This approach is known as stratified area random sampling and it is not something most people have the time/money/resources to implement.

    The usual situation is that you have 1000 customers and you have one of two choices with respect to surveying: mail or online. According to what I’ve read responses to mail surveys run in the 5 to 10% range and in the 20-30% range for online. While these numbers are described as typical and are in accordance with some of the work I’ve done the fact remains “the results of your survey will vary” and the response rate can be much less. While low response rates are a problem a bigger issue is the self selection aspect of a survey of this type. Self selection isn’t random and the potential for sample bias is very large.

    Under these circumstances what you need to do is make sure, either by asking direct questions on the survey, or by specialized coding of the survey, that you have relevant demographic data on those individuals who did respond. Given that you have this you can summarize whatever you get in terms of the demographics. If you are fortunate and the span of the demographics of the responses is “reasonably” inclusive then there is a reasonable chance that whatever you find will apply to your customers in general. On the other hand, if the assessment of the demographics of your self selected sample is not representative then you will have to view the findings of your survey accordingly.

    #169010 Reply

    OK…I think I understand the idea of making sure that your demographic strata are representative of the population. Assuming we cover our demographic bases, is there then a rule of thumb or calculation for determining the percentage of returned surveys we would need to get back to have confidence in some of the descriptive data we would be calculating?

    #169011 Reply

    DanBart wrote:

    OK…I think I understand the idea of making sure that your demographic strata are representative of the population. Assuming we cover our demographic bases, is there then a rule of thumb or calculation for determining the percentage of returned surveys we would need to get back to have confidence in some of the descriptive data we would be calculating?

    Dan: You were lucky enough to get Robert to provide you guidance on what needs to be done to ensure a proper sample is developed. However, you won’t get the rest of us to do your homework for you.

    #169012 Reply

    OK, I think I get it.

    This really is not a forum focused at getting tangible results. It’s just a place where people come to play games of intellectual one-upsmanship. Coming here for assistance , MBBinWI, is part of doing my homework. If you don’t know the answer, which it appears neither of you do, save me the time by either not responding or cutting down on the amount of space it takes to tell me something my high-school aged son has already figured out in his AP stats class.

    Last time I’ll be using these forums as a resource expecting to get any practical advice…just reinforces why LSS is dying on the vine. Practitioners are few and far, theorists are all around us.

    #169013 Reply

    DanBart wrote:

    OK, I think I get it.

    This really is not a forum focused at getting tangible results. It’s just a place where people come to play games of intellectual one-upsmanship. Coming here for assistance , MBBinWI, is part of doing my homework. If you don’t know the answer, which it appears neither of you do, save me the time by either not responding or cutting down on the amount of space it takes to tell me something my high-school aged son has already figured out in his AP stats class.

    Last time I’ll be using these forums as a resource expecting to get any practical advice…just reinforces why LSS is dying on the vine. Practitioners are few and far, theorists are all around us.

    No, you really don’t seem to get it. You come here and throw out your dirty laundry expecting someone to clean it for you.

    Had you come with a proposal as to what you think you should do, and some rationale as to why you think that is right, you would have received more meaningful responses (and what you got from Robert was way more than you deserved considering the original post).

    Part of the reason that SS fails in companies like yours is attitudes like yours. Let me dump my problem on someone else. Do some due diligence in trying to find the answer, then you can post it here and get meaningful feedback. Throw out your dirty laundry and we’ll throw it back at you.

    #169014 Reply
    Avatar of Robert Butler
    Butler
    Member
    Reputation - 0
    Rank - Aluminum

    I gave you the short, correct answer to your question in the first sentence of my initial response. What I was attempting to do in the rest of that post was provide you with some understanding of the problem you faced. I also tried to give you some sense of what you could do given your situation. Since you found this attempt to be offensive please accept my apologies.

    What I will offer is that your second comment/question “Assuming we cover our demographic bases, is there then a rule of thumb or calculation for determining the percentage of returned surveys we would need to get back to have confidence in some of the descriptive data we would be calculating?” misses the point.

    The short answer to the question part of the above is – no. The additional thoughts below address the first part – the assumption of coverage and the ramifications of the possible responses.

    Unless you have control over the survey process in a form such as a stratified area random sample you are going to get whatever you get with no guarantee of demographic coverage. If you have demographic information then you can post-hoc parse the responses from your self selected sample along the lines of the demographics, see what you have, and, from there, decide if you have enough to use for whatever purpose you had in mind.

    For example.

    You have 1000 customers. Before sending out the questionnaire you think they fall into 4 specific groups with a distribution of 100, 400, 250 and 250.

    You send out the questionnaire and you get back the following:

    10/100, 10/400, 25/250, 50/250 for return rates of 10%, 2.5%, 10% and 25%

    Your question is: are these returns enough to give you some confidence in using the results to run calculations/estimations/etc. The short answer is yes. Can you quantify your degree of confidence – other that a simple reporting of percent returns, in the absence of any other information, probably not. Is there a general rule of thumb with respect to such things – no. Is what you have better than nothing – absolutely.

    To highlight the lack of a general rule consider the following: 10/400 sounds pretty thin but what if those 10 represent over 70% of our sales to that demographic? Is this enough of a sample or isn’t it?

    It depends – If 70% of our CURRENT sales were represented in the 10 response from the 400 …it could mean that the 390 non-respondents are small potatoes of no consequence and we can base our decisions on what the 10 responded and not worry.

    Of course it could mean something else: Maybe the 390 are the future of that demographic and, since we have zero input from them any moves we make to address the current 10 will, at best meet there needs and at worst drive the 390 elsewhere and ultimately lose us that demographic.

    #169015 Reply
    Avatar of Robert Butler
    Butler
    Member
    Reputation - 0
    Rank - Aluminum

    Typo: That should have been 10%, 2.5%, 10% and 20%

Viewing 8 posts - 1 through 8 (of 8 total)

Register Now

  • Stop this in-your-face notice
  • Reserve your username
  • Follow people you like, learn from
  • Extend your profile
  • Gain reputation for your contributions
  • No annoying captchas across site
And much more! C'mon, register now.

Reply To: Surveys – How to determine representative sample
Your information:




Login Form