Six Sigma Project Tests
Six Sigma – iSixSigma › Forums › Old Forums › General › Six Sigma Project Tests
- This topic has 13 replies, 7 voices, and was last updated 13 years, 1 month ago by
chhabra.
- AuthorPosts
- January 4, 2008 at 9:32 am #49032
I am doing a project on students feedback of a class. There are 15 students in the batch, There was a feed back taken 3 months earlier which shows a rating of 52%. The min should have been 70%. After corrective action taken another feedback was taken last week which shows rating of 78%. Now using the six sigma tests, which test should i use to prove that there was a significant improvement after taking corrective actions. If i am using hypothesis tests for proportions using p test then what should the number of successes x be?
0January 4, 2008 at 12:32 pm #166799
WestfieldMember@WestfieldInclude @Westfield in your post and this person will
be notified via email.Firstly, congratulations on achieving such an impressive improvement!
Sorry to sound cautious, though, but don’t forget that correlation doesn’t equal causality. Put another way, can you demonstrate statistically that the improved rating is definitely related to the improvement effort, and not to some other factor (e.g. the very fact that you are running the survey can change people’s perspective; or growth over time in the students’ confidence in the class topic; or a ‘feel-good’ factor such as who won the big match the night before.)
I too would be interested to know what tests people might suggest that would add validity to the (probably justifiable) assumption that there is a causality link that you can claim.
Good luck!
0January 4, 2008 at 1:08 pm #166805Westfield – great comments,
Some other ideas… There are many pitfalls when using information from surveys, especially when you are using the survey more than once with the same population over time.Assuming you have taken the time to construct a valid survey, which is beyond the scope of this discussion, two thoughts come to mind. First, if you have the capability, you can use the same survey over time with different populations, and then compare them. I know this will take significantly more time, but you have a better chance of proving that the corrective actions have a causal relationship.
The second thought, is that you could also compare performance of the students after the class. Ultimately, students should be able to perform whatever they have been taught, and if you can link performance improvement with corrective actions, then you can prove that your changes made a significant impact on performance.
Hope this helps.0January 5, 2008 at 7:07 am #166864Hi Westfield,
The project which i am doing is based on student satisfaction of all the faciltities at the centre/institute. If the students are satisfied then through their good will they will ask their friends to enroll in this institute, thus in turn increasing the revenue of the centre/institute.
For this we had conduted a feedback of 15 students of a class 3 months ago to check the satisfaction level. with the following points Faculty Knowledge, Presentation Skills, Puntuality, Machine Availability, Lab Instructor availability, Internet connectivity, Books given to students (courseware), Library Books.
Each point had to be given marks from 2 to 10 (2 been least and 10 mac). The min satisfaction level identified was 70%.
In the 1st feedback we had a aggregate satisfaction level of 54.36% which was very below the min level. To improve the satisfaction level we took all corrective actions for the above points.
Another feedback was taken last week with the same 15 students and covering the same points. Now the aggregate satisfaction level has shot up to 72.18%.and the revenue by 20%.
My question is how do i prove it statistically that there was sufficient improvement. What statistical test should i use to prove it. If i am using the proportion test (p test) because the results are in percentage, then what value should be x in the formula.
Kindly help me in this as i am not able to prove it statistically.
Thanks in advance
amit
0January 5, 2008 at 9:12 am #166865
KrishnamParticipant@KrishnamInclude @Krishnam in your post and this person will
be notified via email.Hi Amit
Please see my comment below
Null Hypotheses (Ho)
Rate of satisfaction levels not change
Alternate Hypothesis (Ha)
Rate of satisfaction level change after corrective action
Minitab Output
Test and CI for Two Proportions
Sample X N Sample p
1 8 15 0.533333
2 11 15 0.733333
Estimate for p(1) – p(2): -0.2
95% CI for p(1) – p(2): (-0.537374, 0.137374)
Test for p(1) – p(2) = 0 (vs not = 0): Z = -1.16 P-Value = 0.245
Conclusions
There is evidence that the rate of satisfaction level was substation ally increased (after corrective action). The new rate of satisfaction level is 72.18% compare to old rate if 54.36%
And also P value indicates 0.245 greater than alpha (0.05) means reject the null hypotheses
Just my opinion.
Regards,
Krishnam
0January 5, 2008 at 10:13 am #166866Amit,
I think, whit this survey, you can only make a statement about the class included in the survey. Hypotheses testing prove improvement is statistically significant, but only in this class.
I understand the goal of the project is to improve Net Promoter Score in the whole institution, not only in one class.
Therefore I suggest you make a random sampling of the students from the whole institution. With discrete data, the sample size should be at least 100.
Or, improvements were a pilot to test impact on one class? Next will you roll out to the rest of the institution?
Just my opinionSilviu
0January 5, 2008 at 10:35 am #166867
Innocent BystanderParticipant@Innocent-BystanderInclude @Innocent-Bystander in your post and this person will
be notified via email.A P value of 0.245 means do NOT reject the null hypothesisThe p-value is the probability of getting data at least as extreme as the observed data if the null hypothesis were true.
0January 5, 2008 at 11:27 am #166868
KrishnamParticipant@KrishnamInclude @Krishnam in your post and this person will
be notified via email.Yes you are correct.
If p value (0.245) greater than commonly choosen alpha (0.05) levels, there is no evidence for a difference in satisfaction levels.
More data is required
Thanks and regards,
Krishnam0January 5, 2008 at 12:06 pm #166869
WestfieldMember@WestfieldInclude @Westfield in your post and this person will
be notified via email.HiGlad someone else corrected the p value interpretation – an easy mistake! I also agree that there are insufficient values to generate useful statistical conclusions.You might want to consider, if only as an exercise, doing a chi square test, as there were several different categories of satisfaction in the survey. This might show whether the proportion of marks awarded differed significantly after the improvements, and therefore whether one improvement effort was especially successful and another especially unseuccessful, leading to further improvement opportunities.However, be careful: are these categories truly independent in the respondents’ minds? And is there a risk that the subjectivity inherent in this sort of survey and the likely influence of other factors may render the results unreliable?My gut feel? You achieved an improvement. Rejoice and move onto the next thing!Good luck
0January 5, 2008 at 6:34 pm #166874
WestfieldMember@WestfieldInclude @Westfield in your post and this person will
be notified via email.I’m glad that someone else picked up on the correct interpretation of the p value, and I agree that more data are required to prove the point you are trying to make.
If only as an opportunity to use the tool, you might try using chi square, since you have a number of categories of satisfaction that you are analysing. This would show whether there was a significant change in the proportion of points allocated, and therefore might indicate whether a specific improvement effort was especially successful. This might then point to a new improvement opportunity. However, the categories of satisfaction are probably not mutually exclusive, so this may only prove partially useful. Also, if the respondents tended to mark each category equally in both surveys, then chi square is unlikely to reveal much of use.
Apart from the small amount of data, I would also remain concerned at the lack of objectivity in this sort of survey, and the fact that other factors which have not been recorded are likely to have impacted the results.
My gut feel? You did achieve an improvement. Rejoice and move on to something else!
Good luck
0January 7, 2008 at 7:34 am #166894hello Krishnam,
Thanks for you solution. I just have one more doubt? When you tested this project on a mini tab how did you get the value of X as 8 and 11?
Thanks in advance
amit
0January 7, 2008 at 6:16 pm #166920
Mike CarnellParticipant@Mike-CarnellInclude @Mike-Carnell in your post and this person will
be notified via email.Amit,
“Student feedback?” They are students why doen’t you check to see if they learned anything since that is why most people attend class.
When instructors know they are being evaluated with a “smiles” test you stop delivering knowledge transfer and deliver adult entertainment.
Just my opinion.0January 8, 2008 at 4:20 am #166943
KrishnamParticipant@KrishnamInclude @Krishnam in your post and this person will
be notified via email.Hi Amit
For your question answer as follows
You know N=15 in the both the cases (total students are 15)
And You know Sample-P ie. % satisfaction level, first time 54.36% for that I have taken X is 8 (calucaleted)
Second time % satisfaction level 72.18% for that I have taken X is 11(calculated)
I hope its calrify to you
Regads,
Krishnam
0January 8, 2008 at 6:59 am #166945Hi Krishnam,
Thanks of the clarification.
Regards,
Amit
0 - AuthorPosts
The forum ‘General’ is closed to new topics and replies.