With that methodology, you wouldn’t be measuring training “effectiveness” (however you wish to define it).
First, your training participants are not entering class with the same baseline understanding of statistics, excel, etc. So their understanding of Minitab tools AFTER class is really going to be the sum of the understanding before class plus what they gained during class.
In addition, with your method, you actually have 20 distinct measurement systems. Regardless of how well you define what a “2” or “3” means, you’re still adding significant measurement system variation. We’re still not considering the fact that your data will almost certainly be clustered around the 3’s and 4’s.
Lastly, does it really matter what they think their proficiency level is immediately after class? Shouldn’t the point of the training be to have them retain the info long enough to implement the tools?
So if you’re really interested in measuring the effectiveness of your training class (as opposed to just making some neat little boxplots to put on a powerpoint), I’d recommend doing a pre-training assessment test to determine baseline knowledge levels. Then I’d follow that up with two post-training assessment tests (final day of training & 30-ish days after training). Then I’d run paired t-tests (pre-training test vs last day test and last day test vs 30 day test) and pareto the the lowest scoring problems for follow-up.
(Disclaimer: I’m deliberately avoiding any discussion regarding whether the test scores would be discrete or continuous)
.
