Testing Random sampling based survey results
Six Sigma – iSixSigma › Forums › Old Forums › General › Testing Random sampling based survey results
 This topic has 11 replies, 10 voices, and was last updated 20 years, 2 months ago by Ward.

AuthorPosts

June 16, 2002 at 2:01 pm #29662
Rajanga SivakumarParticipant@RajangaSivakumar Include @RajangaSivakumar in your post and this person will
be notified via email.We have done a survey based on stratified random sampling a population of about 12000 spread in 11 locations for a 95% confidence level and 2% margin error. The required sampling number was about 2000 and we received over 2100 responses. The questions were be responded on a five point scale.The results have been determind as %favourable, %neutral and %unfavourable for every question, specific groups of questions, by location, by department etc. etc. My question is on as to how to veify, if indeed the results are acceptable within the norms specified. Is this known as hypothesis testing? How is it done? Is it a must? Are there any programs available in an excel spread sheet which can be used to get the required results. From my questions it would be apparent that my knowledge in statistics is limited. Please help with some references where possible. You may available in soft version to my email id below. Thanks.
[email protected]
0June 18, 2002 at 9:11 am #76484
Rajanga SivakumarParticipant@RajangaSivakumar Include @RajangaSivakumar in your post and this person will
be notified via email.Can some one help please? I have not seen any postings to my questions in the below mentioned message of 16th June. Thanks for your help.
Rajanga0June 18, 2002 at 3:11 pm #76492Have you done a measurment system analysis? You need to determine what percentage of your sampling error is due to variation within your measurment system. Yes, this is necessary. Otherwise you could be working with useless information.
0June 19, 2002 at 8:16 am #76507
Rajanga SivakumarParticipant@RajangaSivakumar Include @RajangaSivakumar in your post and this person will
be notified via email.Jamie, thanks for your response.Our measurement process starts with manual data entry into an access data base from physically filled in paper questionnaires from the various employees. Data entry is validated by manual sample checks and examination. The next part is the query generation for the results (# of people answeing as favourable, neutral or unfavourable) for the various survey parameters (like management, communication, relationship etc,) by location, department etc. This information is transposed from access database to an excel file. Here there are built in checks to ensure that the numbers are valid (like the three % should add to100% etc). What other things should I look for in the measurment system analysis? How do I determine what percentage of the sampling error is due to variation within this measurment system? As I mentioned our random sampling number was determinrd from the total population for 95% confidence level and 2% margin error by using the standard table. Please help.
[email protected]
0June 19, 2002 at 1:31 pm #76511Hi,
Once i did the similar kind of survey in our organization. During that time i tested for the significant differences between different plots. At that time i used ANOVA and also tested each pair of buildings independently using some non paramteric test.
Disadvantage of using ANOVA is that it has an assumption that data should follow normal distribution, which is very difficult because our scale is limited.
By using independet non parametric test it will reduce our confidence level.
But i used both ANOVA and also tested with non paramtric tests also. I dont know whether it is right or not, but i got good results from these analysis.
I used MINITAB for doing the above.
Hope this is going to help u . If need any mroe information contact me at [email protected]
thanks
A.Sridhar
0June 20, 2002 at 6:59 am #76520
Norbert DickelParticipant@NorbertDickel Include @NorbertDickel in your post and this person will
be notified via email.I would not worry too much about the normality of the population when you do an ANOVA analysis. Yes, you are correct that ANOVA expects a normal distribution but the results are not highly sensitive to this requirement. If your population is showing an approximate hauss curve, then you should be fine. If not, be aware that there are two nonparametric test available too: the KrushallWallis test and the Mood’s Median Test.
0June 20, 2002 at 12:40 pm #76528
jayne WilliamsParticipant@jayneWilliams Include @jayneWilliams in your post and this person will
be notified via email.Have you tried MiniTab Statistical software – as this will not only generate a sample number required in order to achieve 95% confidence, as well as being able to use hypothesis testing (results are in several differnt formats – depending on what test you require). Default is set for the P value. I suggest you go to http://www.Minitab.com or try statistica enclyclopedia on the web for any stats help.
0June 20, 2002 at 3:03 pm #76538
Andrew M. BrodyParticipant@AndrewM.Brody Include @AndrewM.Brody in your post and this person will
be notified via email.You may want to contact the ASQ to see if they will let you benchmark off how they do their salary surveys.
Andrew M. Brody0June 20, 2002 at 3:59 pm #76541Rajanga,
The generally accepted statistical test for analyzing survey data is the KruskalWallis Test. However, this test requires that the subgroups being compared to have equal variances, as does the ANOVA test. Mood’s Median Test likes to see similar shapes, which is usually not the case for survey data. If you fail both of the assumptions (Normality and Equal Variances), then the 2sample t test is the most robust of the continuous type of tests.
Another point of discussion is the 15 scale that is being utilized cannot pass the MSA resolution requirements. This will have a direct effect on the test of normality. In general, the 17 or 19 scales should be considered in the future due to this limitation.
Another approach would be to review the discrete data tests (ChiSquare contingency table and 2Proportion tests), since your data is actually discrete.
Overall, you should be performing Power of the Test to validate that when making comparisons that you have enough data (Beta Error). There are standard tests for ANOVA, 2sample t and 2proportion. The ANOVA test can be utilized if you are able to transform your data to normal distributions.
I hope that this helps. All the best in your analysis!0June 20, 2002 at 6:12 pm #76543
Dave Davidson Methot, PhDParticipant@DaveDavidsonMethot,PhD Include @DaveDavidsonMethot,PhD in your post and this person will
be notified via email.Some of the responses so far have been helpful, such as those recommeding the KruskalWallis Test. However, the core issue hasn’t been addressed. That is, since the results were on a Likerttype five point scale, the survey data is ordinal rather than interval or ratio. Therefore, you can’t use parametric tests such as ANOVA or ttests, which assume equally spaced (“intervals” hence the name) data points. Thus, you are limited to nonparametric tests. Since you are comparing multiple locations on the same variables, the KruskalWallis is the correct choice of test. Hope this helps a bit more.
0June 21, 2002 at 8:17 pm #76570more than that, for the information provided it appears that the data was further parsed into 3 catagories and counted. Comparison is then between some expected count value and a projected (or expected) count dertermined by some predetermined specification. Depending upon what the hypothesis which is being tested, a simple chisqared analysis might be the best …
0June 24, 2002 at 10:22 pm #76611Some good pionts have been made but no one has discussed the anaylsis of the data. By catagorizing the results into the three catagories; percent favorable, unfavorable and average you are giving away information and making the data even more discrete.
The question I have is that with 2100 responses can’t one take advantage of central limit theorm and analyze the average and standard deviation of the whole and the stratified groups. I would hope for at least 30 responses from any one stratified group to do this and the more the better. By doing this you can start to analyze the averages responses of the stratified subgroups like it was continuous data and use the ANOVA and Ttests. This is imperfect but is simple. A random response for a given question will result in an average of 3 and standard deviation of 1. If there is response that is not random the standard deviation will be much lower that 1. To test if the average response is different from 3 a 1 sample Ttest can be done on the average and standard deviation. 2 Sample Ttests can be done to compare different stratified groups. As pointed out in the MSA comments the biggest problem will probably be in the area of discrimination. However averaging should help in this since the average of large subgroups is being analized not the individual responses. If a 7 or 9 point scale were used the results would have been closer to continous data and discrimination issue smaller.
Comments please.
PS I am not highly familar with Kruskal Wallis so will read up in it. Anyone have a good reference?
0 
AuthorPosts
The forum ‘General’ is closed to new topics and replies.