Trying to Determine the Health of a Knowledge Improvement Project

Six Sigma – iSixSigma Forums General Forums Implementation Trying to Determine the Health of a Knowledge Improvement Project

Viewing 6 posts - 1 through 6 (of 6 total)
  • Author
  • #55556

    Boyan Angelov

    Hi all,

    I am working on Knowledge improvement project in a Software Support organization.
    We currently have a large DB of documents which our customers may use to solve problems without opening Support case.
    This DB is constantly updated and modified to reflect the latest product changes.

    The correlation coefficient between Number of Document Views per month (independent) and Number of opened cases per month(dependent ) is p=0.71.
    I interpret this coefficient value as that our customers are not finding the needed information and are opening tickets.

    Can you please help me determine:
    – Is my thought about the coefficient valid?
    – How an improvement can be detected?- by reversing the coefficient to a negative trend , maybe?

    I have all kinds of metrics, cases opened, cases closed, document views, documents created , documents revised etc. all details are per month.

    PS. In case you have experience in the area, I will be grateful if you can share some suggestions.



    Robert Butler

    Based on your description it doesn’t sound like your correlation coefficient has any physical meaning. The fact that as the number of document views has increased over time the number of opened cases per month has also increased is interesting but there is no guarantee that the correlation is indicative of cause and effect.

    If you want to test this hypothesis you will have to take the time to quantify your metrics. For example – what kind of cases, what is the frequency distribution of case type over time, what kind of documents are being viewed, what kind of cases are most likely associated with a particular kind of document, etc.

    As for an interpretation above and beyond what was stated in my first paragraph – it could be many things. Maybe the document searches are very effective and, as a result people open cases because they are interested in what they have found and want to know if you have more information on the subject but under a different index.

    It could be that the issue is the continuous updating and modifying of the DB resulting in people having to constantly re-learn how to navigate through the system. It could be that your interface has become too cumbersome due to the modifications you have made, etc.



    @bangelov – I concur with @rbutler. Do you have a way to identify those who search the database and then open a ticket? This would be a better correlation on the usefulness of the database. Also, do you have any metrics on whether the open tickets are addressed by support personnel that find the answer in the database? This would also help to identify that the info in the database is not sufficiently easy for the novice user to find the needed info.

    I often use similar databases to find solutions to automobile, computer, and appliance issues. I am motivated to find the answer myself to save money by fixing the issue myself. If your database is difficult to use, and there is little cost for the user to just open a ticket and shift the responsibility to someone else, that may also be a factor.


    Boyan Angelov

    Thank you both for sharing your thoughts and for the metrics that you suggested!
    I do not have much additional details that can do the split per customer/case type etc.
    I’m currently trying to gather additional details and continue the investigation further.



    Hello Boyan,

    Coul it also possibly be that your support personnel often use/access/refer to the DB to solve issues relate to ticket queries. In that case, your p will always be high!
    Just a thought.


    Chris Seider

    1. Remember that a relationship doesn’t mean that there’s cause and effect.
    2. There may not even be a relationship but you can’t quote the correlation coefficient solely. You need to see what the p-value is for statistical confidence.
    3. Is the data properly measured? If not…would cause the r value to drop.

    Just a few extra thoughts.

Viewing 6 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.