iSixSigma

Testing Random sampling based survey results

Six Sigma – iSixSigma Forums Old Forums General Testing Random sampling based survey results

Viewing 12 posts - 1 through 12 (of 12 total)
  • Author
    Posts
  • #29662

    Rajanga Sivakumar
    Participant

    We have done a survey based on stratified random sampling a population of about 12000 spread in 11 locations for a 95% confidence level and 2% margin error. The required sampling number was about 2000 and we received over 2100 responses. The questions were be responded on a five point scale.The results have been determind as %favourable, %neutral and %unfavourable for every question, specific groups of questions, by location, by department etc. etc. My question is on as to how to veify,  if indeed the results are acceptable within  the norms specified. Is this known as hypothesis testing? How is it done? Is it a must? Are there any programs available in an excel spread sheet which can be used to get the required results. From my questions it would be apparent that my knowledge in statistics is limited. Please help with some references where possible. You may available in soft version to my email id below. Thanks.
    [email protected]
     

    0
    #76484

    Rajanga Sivakumar
    Participant

    Can some one help please? I have not seen any postings to my questions in the below mentioned message of 16th June. Thanks for your help.
    Rajanga

    0
    #76492

    Jamie
    Participant

    Have you done a measurment system analysis?  You need to determine what percentage of your sampling error is due to variation within your measurment system.  Yes, this is necessary.  Otherwise you could be working with useless information.

    0
    #76507

    Rajanga Sivakumar
    Participant

    Jamie, thanks for your response.Our measurement process starts with manual data entry into an access data base from physically filled in paper questionnaires from the various employees. Data entry is validated by manual sample checks and examination. The next part is the query generation for the results (# of people answeing as favourable, neutral or unfavourable) for the various survey parameters (like management, communication, relationship etc,) by location, department etc. This information is transposed from access database to an excel file. Here there are built in checks to ensure that the numbers are valid (like the three % should add to100% etc). What other things should I look for in the measurment system analysis?  How do I determine what percentage of the sampling error is due to variation within this measurment system? As I mentioned our random sampling number was determinrd from the total population for 95% confidence level and 2% margin error by using the standard table. Please help.
    [email protected]
     

    0
    #76511

    Sridhar
    Member

    Hi,
    Once i did the similar kind of survey in our organization. During that time i tested for the significant differences between different plots. At that time i used ANOVA and also tested each pair of buildings independently using some non paramteric test.
    Disadvantage of using ANOVA is that it has an assumption that data should follow normal distribution, which is very difficult because our scale is limited.
    By using independet non parametric test it will reduce our confidence level.
    But i used both ANOVA and also tested with non paramtric tests also. I dont know whether it is right or not, but i got good results from these analysis.
    I used MINITAB for doing the above.
    Hope this is going to help u . If need any mroe information contact me at [email protected]
    thanks
    A.Sridhar
     

    0
    #76520

    Norbert Dickel
    Participant

    I would not worry too much about the normality of the population when you do an ANOVA analysis. Yes, you are correct that ANOVA expects a normal distribution but the results are not highly sensitive to this requirement. If your population is showing an approximate hauss curve, then you should be fine. If not, be aware that there are two nonparametric test available too: the Krushall-Wallis test and the Mood’s Median Test.

    0
    #76528

    jayne Williams
    Participant

    Have you tried MiniTab Statistical software – as this will not only generate a sample number required in order to achieve 95% confidence, as well as being able to use hypothesis testing (results are in several differnt formats – depending on what test you require).  Default is set for the P value.  I suggest you go to http://www.Minitab.com or try statistica enclyclopedia on the web for any stats help.
     
     

    0
    #76538

    Andrew M. Brody
    Participant

    You may want to contact the ASQ to see if they will let you benchmark off how they do their salary surveys.
    Andrew M. Brody

    0
    #76541

    JAD
    Participant

    Rajanga,
    The generally accepted statistical test for analyzing survey data is the Kruskal-Wallis Test.  However, this test requires that the subgroups being compared to have equal variances, as does the ANOVA test.  Mood’s Median Test likes to see similar shapes, which is usually not the case for survey data.  If you fail both of the assumptions (Normality and Equal Variances), then the 2-sample t test is the most robust of the continuous type of tests. 
    Another point of discussion is the 1-5 scale that is being utilized cannot pass the MSA resolution requirements.  This will have a direct effect on the test of normality.  In general, the 1-7 or 1-9 scales should be considered in the future due to this limitation.
    Another approach would be to review the discrete data tests (Chi-Square contingency table and 2-Proportion tests), since your data is actually discrete. 
    Overall, you should be performing Power of the Test to validate that when making comparisons that you have enough data (Beta Error).  There are standard tests for ANOVA, 2-sample t and 2-proportion.  The ANOVA test can be utilized if you are able to transform your data to normal distributions.
    I hope that this helps.  All the best in your analysis!

    0
    #76543

    Dave Davidson Methot, PhD
    Participant

    Some of the responses so far have been helpful, such as those recommeding the Kruskal-Wallis Test. However, the core issue hasn’t been addressed. That is, since the results were on a Likert-type five point scale, the survey data is ordinal rather than interval or ratio. Therefore, you can’t use parametric tests such as ANOVA or t-tests, which assume equally spaced (“intervals” hence the name) data points. Thus, you are limited to non-parametric tests. Since you are comparing multiple locations on the same variables, the Kruskal-Wallis is the correct choice of test. Hope this helps a bit more.

    0
    #76570

    Ted
    Member

    more than that, for the information provided it appears that the data was further parsed into 3 catagories and counted.  Comparison is then between some expected count value and a projected (or expected) count dertermined by some predetermined specification.  Depending upon what the hypothesis which is being tested, a simple chi-sqared analysis might be the best … 

    0
    #76611

    Ward
    Participant

    Some good pionts have been made but no one has discussed the anaylsis of the data.  By catagorizing the results into the three catagories; percent favorable, unfavorable and average you are giving away information and making the data even more discrete.  
    The question I have is that with 2100 responses can’t one take advantage of central limit theorm and analyze the average and standard deviation of the whole and the stratified groups.  I would hope for at least 30 responses from any one stratified group to do this and the more the better.  By doing this you can start to analyze the averages responses of the stratified subgroups like it was continuous data and use the ANOVA and T-tests.  This is imperfect but is simple.  A random response for a given question will result in an average of 3 and standard deviation of 1.  If there is response that is not random the standard deviation will be much lower that 1.  To test if the average response is different from 3 a 1 sample T-test can be done on the average and standard deviation.   2 Sample T-tests can be done to compare different stratified groups.  As pointed out in the MSA comments the biggest problem will probably be in the area of discrimination.   However averaging should help in this since the average of large subgroups is being analized not the individual responses.  If a 7 or 9 point scale were used the results would have been closer to continous data and discrimination issue smaller.
    Comments please.
    PS I am not highly familar with Kruskal Wallis so will read up in it.  Anyone have a good reference?
     

    0
Viewing 12 posts - 1 through 12 (of 12 total)

The forum ‘General’ is closed to new topics and replies.