Finding a P-value on a Small Limited Range of Outputs from a Survey
- February 25, 2018 at 9:37 am #55946
It’s been a while since I’ve had to do something like this. I have over 400 surveys done by people. The questions use a scale of 1 to 5 for their answers. I’m concerned that since there is only a range of discrete values of 1 to 5 that using a p-value (a correlation Pearson Bivariate) test would not give an accurate or good p-value. Or since the range is so limited (small) it won’t accurately show if two questions actually correlate or not.
One question asks How often have you felt stressed in the last week (1 never to 5 very often). Another questions asks All in all, I am inclined to feel that I am a failure (1 strongly disagree to 5 strongly agree). So my alternate hypothesis is there the more a person is to agree that they feel like a failure (answer a number closer to 5) the more likely they will feel stress more often (answer closer to 5). Where my null Hypothesis is there is no correlation. I have already done a Correlation (Pearson) Bivariate test and the p-value I got was 0.000. But, again I’m concerned that is too low of a p-value and I feel I got it more because there are such a limited output for the survey answers. I think of course it’s such a small p-value because any small correlation in this limited range would be “force” a p-value to be small. I hope this makes sense and I hope to get some feedback. Thanks in advanceFebruary 25, 2018 at 12:35 pm #202306
So an update. I have found out I need to be using a Spearman Correlation Bivariate test not the Pearson. I’m still getting out p-values of 0.000 or they are extremely in line with each other. I feel this is still incorrect.February 25, 2018 at 3:02 pm #202307
One thing to consider with this sort of survey is that data are qualitative, not quantitative. Have you done chi-square or multivariate analysis?February 25, 2018 at 3:48 pm #202309
I guess the bigger question is why is it that you “feel” this is incorrect because you have a limited range? This doesn’t make sense. If that was the case then no one would run studies where the responses were binary yes/no.
My experience with Likert data has been that there is no real issue with respect to regressing one Likert rating against another. You can also look at a plot of the the two responses against one another but you will need to have a plotting package that will allow you to jitter the data otherwise you will just have a bunch of single points on a plot that won’t tell you much of anything.
As for your feelings, @Straydog has made a point which, if you expand it slightly, might assuage those feelings. Treat the data as “top box” and see if the chi-square results are in line with what you are seeing. Top box will treat either 5 or 4 and 5 as a 1 and everything else as a 0. Use this conversion and analyze the results as a simple 2×2 table.February 27, 2018 at 7:47 pm #202329
@Straydog since I’m working with survey questions administered to the same group of people and they are not independent groups I don’t think I can use the chi-square test. I did the same Correlation (Spearman) Bivariate test using R and got the following results. (I hope the picture loads) since I got a negative rho value I’m happy that the type of relationship lines up with my hypothesis. I feel I’m limited to doing just this test to the data (which is the same group and ordinal data), if I’m incorrect please let me know. @rbutler
Also my p-value from the R output is extremely small as opposed to zero so that was beneficial.
You must be logged in to reply to this topic.