iSixSigma

Six Sigma Project Tests

Six Sigma – iSixSigma › Forums › Old Forums › General › Six Sigma Project Tests

Viewing 14 posts - 1 through 14 (of 14 total)
  • Author
    Posts
  • January 4, 2008 at 9:32 am #49032

    chhabra
    ★★★★★★★★★★
    Participant

    I am doing a project on students feedback of a class. There are 15 students in the batch, There was a feed back taken 3 months earlier which shows a rating of 52%. The min should have been 70%. After corrective action taken another feedback was taken last week which shows rating of 78%. Now using the six sigma tests, which test should i use to prove that there was a significant improvement after taking corrective actions. If i am using hypothesis tests for proportions using p test then what should the number of successes x be?

    0
    January 4, 2008 at 12:32 pm #166799

    Westfield
    ★★★★★★★★★★
    Member

    Firstly, congratulations on achieving such an impressive improvement! 
    Sorry to sound cautious, though, but don’t forget that correlation doesn’t equal causality. Put another way, can you demonstrate statistically that the improved rating is definitely related to the improvement effort, and not to some other factor (e.g. the very fact that you are running the survey can change people’s perspective; or growth over time in the students’ confidence in the class topic; or a ‘feel-good’ factor such as who won the big match the night before.)
    I too would be interested to know what tests people might suggest that would add validity to the (probably justifiable) assumption that there is a causality link that you can claim.
    Good luck!
     
     
     

    0
    January 4, 2008 at 1:08 pm #166805

    Aquinas
    ★★★★★★★★★★
    Member

    Westfield – great comments,
    Some other ideas… There are many pitfalls when using information from surveys, especially when you are using the survey more than once with the same population over time.Assuming you have taken the time to construct a valid survey, which is beyond the scope of this discussion, two thoughts come to mind. First, if you have the capability, you can use the same survey over time with different populations, and then compare them. I know this will take significantly more time, but you have a better chance of proving that the corrective actions have a causal relationship.
    The second thought, is that you could also compare performance of the students after the class. Ultimately, students should be able to perform whatever they have been taught, and if you can link performance improvement with corrective actions, then you can prove that your changes made a significant impact on performance.
    Hope this helps.

    0
    January 5, 2008 at 7:07 am #166864

    chhabra
    ★★★★★★★★★★
    Participant

    Hi Westfield,
    The project which i am doing is based on student satisfaction of all the faciltities at the centre/institute. If the students are satisfied then through their good will they will ask their friends to enroll in this institute, thus in turn increasing the revenue of the centre/institute.
    For this we had conduted a feedback of 15 students of a class 3 months ago to check the satisfaction level. with the following points Faculty Knowledge, Presentation Skills, Puntuality, Machine Availability, Lab Instructor availability, Internet connectivity, Books given to students (courseware), Library Books.
    Each point had to be given  marks from 2 to 10 (2 been least and 10 mac). The min satisfaction level identified was 70%.
    In the 1st feedback we had a aggregate satisfaction level of 54.36% which was very below the min level. To improve the satisfaction level we took all corrective actions for the above points.
    Another feedback was taken last week with the same 15 students and covering the same points. Now the aggregate satisfaction level has shot up to 72.18%.and the revenue by 20%.
    My question is how do i prove it statistically that there was sufficient improvement. What statistical test should i use to prove it. If i am using the proportion test (p test) because the results are in percentage,  then what value should be x in the formula.
    Kindly help me in this as i am not able to prove it statistically.
    Thanks in advance
    amit
     

    0
    January 5, 2008 at 9:12 am #166865

    Krishnam
    ★★★★★★★★★★
    Participant

    Hi Amit
    Please see my comment below
    Null Hypotheses (Ho)
    Rate of satisfaction levels not change
     
    Alternate Hypothesis (Ha)
    Rate of satisfaction level change after corrective action
     
    Minitab Output
     
    Test and CI for Two Proportions
     
    Sample      X      N  Sample p
    1           8     15  0.533333
    2          11     15  0.733333
     
    Estimate for p(1) – p(2):  -0.2
    95% CI for p(1) – p(2):  (-0.537374, 0.137374)
    Test for p(1) – p(2) = 0 (vs not = 0):  Z = -1.16  P-Value = 0.245
     
    Conclusions
    There is evidence that the rate of satisfaction level was substation ally increased (after corrective action). The new rate of satisfaction level is 72.18% compare to old rate if 54.36%
     
    And also P value indicates 0.245 greater than alpha (0.05) means reject the null hypotheses 
     
     
    Just my opinion.
     
    Regards,
    Krishnam 
     

    0
    January 5, 2008 at 10:13 am #166866

    Silviu
    ★★★★★★★★★★
    Member

    Amit,
    I think, whit this survey, you can only make a statement about the class included in the survey. Hypotheses testing prove improvement is statistically significant, but only in this class.
    I understand the goal of the project is to improve Net Promoter Score in the whole institution, not only in one class.
    Therefore I suggest you make a random sampling of the students from the whole institution. With discrete data, the sample size should be at least 100.
    Or, improvements were a pilot to test impact on one class? Next will you roll out to the rest of the institution?
    Just my opinion

    Silviu 

    0
    January 5, 2008 at 10:35 am #166867

    Innocent Bystander
    ★★★★★★★★★★
    Participant

    A P value of 0.245 means do NOT reject the null hypothesisThe p-value is the probability of getting data at least as extreme as the observed data if the null hypothesis were true.

    0
    January 5, 2008 at 11:27 am #166868

    Krishnam
    ★★★★★★★★★★
    Participant

    Yes you are correct.
    If p value (0.245) greater than commonly choosen alpha (0.05) levels, there is no evidence for a difference in  satisfaction levels.
    More data is required
    Thanks and regards,
    Krishnam

    0
    January 5, 2008 at 12:06 pm #166869

    Westfield
    ★★★★★★★★★★
    Member

    HiGlad someone else corrected the p value interpretation – an easy mistake! I also agree that there are insufficient values to generate useful statistical conclusions.You might want to consider, if only as an exercise, doing a chi square test, as there were several different categories of satisfaction in the survey. This might show whether the proportion of marks awarded differed significantly after the improvements, and therefore whether one improvement effort was especially successful and another especially unseuccessful, leading to further improvement opportunities.However, be careful: are these categories truly independent in the respondents’ minds? And is there a risk that the subjectivity inherent in this sort of survey and the likely influence of other factors may render the results unreliable?My gut feel? You achieved an improvement. Rejoice and move onto the next thing!Good luck

    0
    January 5, 2008 at 6:34 pm #166874

    Westfield
    ★★★★★★★★★★
    Member

    I’m glad that someone else picked up on the correct interpretation of the p value, and I agree that more data are required to prove the point you are trying to make.
    If only as an opportunity to use the tool, you might try using chi square, since you have a number of categories of satisfaction that you are analysing. This would show whether there was a significant change in the proportion of points allocated, and therefore might indicate whether a specific improvement effort was especially successful. This might then point to a new improvement opportunity. However, the categories of satisfaction are probably not mutually exclusive, so this may only prove partially useful. Also, if the respondents tended to mark each category equally in both surveys, then chi square is unlikely to reveal much of use.
    Apart from the small amount of data, I would also remain concerned at the lack of objectivity in this sort of survey, and the fact that other factors which have not been recorded are likely to have impacted the results.
    My gut feel? You did achieve an improvement. Rejoice and move on to something else!
    Good luck
     
     
     

    0
    January 7, 2008 at 7:34 am #166894

    chhabra
    ★★★★★★★★★★
    Participant

    hello Krishnam,
    Thanks for you solution. I just have one more doubt? When you tested this project on a mini tab how did you get the value of X as 8 and 11?
    Thanks in advance
    amit
     

    0
    January 7, 2008 at 6:16 pm #166920

    Mike Carnell
    ★★★★★★★★★★
    Participant

    Amit,
    “Student feedback?” They are students why doen’t you check to see if they learned anything since that is why most people attend class.
    When instructors know they are being evaluated with a “smiles” test you stop delivering knowledge transfer and deliver adult entertainment.
    Just my opinion.

    0
    January 8, 2008 at 4:20 am #166943

    Krishnam
    ★★★★★★★★★★
    Participant

    Hi Amit
    For your question answer as follows
    You know N=15 in the both the cases (total students are 15)
    And You know Sample-P ie. % satisfaction level, first time 54.36% for that I have taken X is 8 (calucaleted) 
    Second time % satisfaction level 72.18% for that I have taken X is 11(calculated)
    I hope its calrify to you
    Regads,
    Krishnam 
     

    0
    January 8, 2008 at 6:59 am #166945

    chhabra
    ★★★★★★★★★★
    Participant

    Hi Krishnam,
    Thanks of the clarification.
    Regards,
    Amit
     

    0
  • Author
    Posts
Viewing 14 posts - 1 through 14 (of 14 total)

The forum ‘General’ is closed to new topics and replies.