iSixSigma

Control Chart for a Sub Group That Varies

Six Sigma – iSixSigma Forums Forum Basics Welcome Control Chart for a Sub Group That Varies

This topic contains 26 replies, has 10 voices, and was last updated by  MBBinWI 7 years, 3 months ago.

Viewing 27 posts - 1 through 27 (of 27 total)
  • Author
    Posts
  • #54003

    Trish

    Putting together a dashboard for a mortgage process. Would like to use a control chart to track average cycle times for applications taken on each date. The number of applications vary each day. Would like opinions of the best chart/way of going about this…thanks

    0
    #192710

    Thomas Whitney
    Member

    Why use such a one dimensional measure such as the average. Plot the data using a box plot by day. So much more information is available and the plot is easy to explain and see things like outliers and dramatic differences in the spread and median of data.

    A note on data collection. I hope you are collecting as many “by” variables as you can such as by office location, person, type of mortgage, city etc. I would love to find out to whom the good outlier data belongs and to whom the bad outlier belongs and find out what is different, That’s Six Sigma Heaven!

    0
    #192712

    MBBinWI
    Participant

    @Trish – in a tool like Minitab, control charts with different sample sizes are no problem. You get U/L control limits that are staggered based on sample size for each individual set of data, but so long as you understand how to interpret these, you should have no problem.
    I agree with Tom that Box/Whisker charts are a much more rich format for the data. Also to collect as many by variables as you can (you may end up being surprised at what is important).

    0
    #192715

    Steve Clapp
    Participant

    I remember learning somewhere that if sample sizes don’t vary by more than 25% of the average sample size, you can just use the average sample size for all x-bar data points. Doing so will result in straight line control limits, which are usually easier to explain to management than jagged control limits. I hope this helps.

    0
    #192716

    Darth
    Participant

    @clapper1 Yeah the first time I saw the 25% rule was in some of the old Ford stuff on control charts.

    @MBBinWI Not sure that Box/Whisker is a richer format just different. While it might show an outlier, is that outlier truly significant? An outlier in a box plot is not the same as an outlier in a control chart. We also want to be careful of slicing and dicing too fine in the event some hypothesis testing wants to be done later. Might lose some Power along the way. In Mini, interval plots and individual value plots are also useful if you want to see a little more detail. Finally, the control chart is not uni-dimensional. There is a lot of info contained in the R or S chart as well.

    0
    #192721

    Chris Seider
    Participant

    Hmmm, a control chart of cycle times? And who would monitor it and respond to the special cause indications?

    Why not pick another metric to monitor that doesn’t wait until the total time of completion so response could happen sooner if the process begins to bog down.

    0
    #192729

    Trish G
    Participant

    Steve, Darth , Chris et al,

    Thanks for all the feedback. I will look at the variation for the subgroups. The 25% thing using the average subgroup size will be a lot easier. I hadn’t heard that before. I have looked at the individual plots but when you get up to 60 or so a day, it gets to look like a Rorschach ink block test.

    The dashboard is being designed for the people who manage the process and is broken into the different units that will be using it; we also established a pipeline measure so that we can react to volume that will begin to grind into our cycle time measure.

    Thanks guys, you’ve been most helpful.

    0
    #192730

    Robert Butler
    Participant

    I’d like to better understand the desire to look at average cycle times and why the OP thinks this would be of any value.

    I know very little about the mortgage application process but given a focus on an average and the variation about that average this would suggest the following:

    1. A mortgage application is a mortgage application. Therefore it doesn’t matter if the application is for a hovel or a hotel complex they are all exactly the same.

    2. Mortgage applications are uniform with respect to property location – same paperwork, same dotting i’s and crossing t’s regardless of municipality or county or state.

    3. When you say “only one channel via phone” what does that mean”?

    4. Why would you care about a control limit on means? It would seem to me that the sample size, as far as a customer is concerned, would be 1 and the issue would be the expected variation in the time needed for my personal mortgage application.

    If you could expand on some of the above perhaps I or someone else might be able to offer additional suggestions.

    0
    #192731

    Gary Cone
    Participant

    @clapper1 @Trish Beware of people who say things like “I remember learning somewhere’. The 25% advice is terrible in light of the tools at our disposal.

    Trish, follow the advice of MBBinWI. If you learn something but are uncomfortable explaining it with a chart with variable limits, deal with it and ask for advice when the time comes.

    0
    #192735

    Robert Butler
    Participant

    …so we have 6 people and we have two kinds of purchases new property and refinance and we have the fact that there is an ebb and flow with respect to incoming applications. I agree with MBBinWI – before you try to set up control charts etc. at the very minimum you would want to look at straight raw plots over time with respect to individuals and mortgage type while also tracking daily incoming and outgoing.

    I would guess that mortgage applications have some kind of seasonal variation so you would want to run the raw plots long enough to make sure you have representative data. Once you have these then you will be in a position to start thinking about dashboards etc.

    0
    #192739

    Trish G
    Participant

    Thanks Paul…When I attempt the EWMA chart in Mini its still asking for a subgroup size..how do I get around this…

    0
    #192740

    Keller
    Participant

    Enter the subgroup size as 1

    0
    #192741

    Trish G
    Participant

    Got it…thanks

    0
    #192745

    Darth
    Participant

    @pkeller@qualityamerica.com

    I don’t agree that the major reason for using an EWMA chart is because of issues of distribution. It is designed to provide more importance to recent data than past data. Anything involving “moving” data will be dampened and thus not quite as responsive. In looking over the cited article from your company website I see that you are probably a bigger fan of Breyfogle than Wheeler since you are fond of mentioning distribution concerns relative to the construction and value of control charts with less than normal shape.

    0
    #192751

    Keller
    Participant

    Darth,
    The EWMA chart has many uses. Its historic use (for which is was originally developed) was in analysis of serially-correlated data. Its use is not restricted to that, and has many practical uses for independent data as well.
    I am a fan of proper data analysis. (period). I respect the original work of Shewhart, and the many well educated professionals who have improved the resource of quality engineering since, including the likes of Ott, Duncan and Montgomery, to name a few with well-established technical pedigree.
    There is some contention within the Quality community that the distribution of both the underlying process and the subgroup averages is irrelevant to the understanding and use of control charts. The debate itself might be viewed as rather esoteric, since both sides would draw similar broad conclusions: the control chart, particularly the X-bar chart, is a useful tool for detecting shifts in a process. The pertinence of the debate, however, is in the details, and has particular impact when applied to other control charts, including the Individual-X chart and the more recently developed CuSum and EWMA charts.

    The argument against the use of probability models to define the control limits includes the following remarks:

    1. Shewhart did not rely upon the Normal Distribution in his development of the control chart; instead, he used empirical (experimental) data, and generated limits that worked for his process.

    2. Since the control chart is not based on a distinct probability model, it is not necessary to fit a distribution or make any assumptions about the process or its data. The control limits that are calculated using the Shewhart equations will always provide control limits that are robust to any differences in the underlying distribution of the process.

    3. If you say that the X-bar chart relies upon the Normal Distribution, you rely upon the Central Limit Theorem. But the Central Limit Theorem would not apply to the subgroup range or sigma calculation anyway, so how do you define limits for the subgroup ranges (or sigma)?

    4. The control limits are set in the “tail areas” of the distribution anyway, so that any attempt to fit a distribution will be subject to errors in these regions.

    The argument for the use of probability models to define the control limits notes the following:

    1. If control charts defined by Shewhart were based entirely on empirical data, and not based on any theory that would have broader implications for all processes, they would be useful for only Shewhart-type processes. This is not the case; the control charts are based upon mathematical (or more precisely, statistical) theory that transcends particular processes.

    2. The control limits are determined mathematically, and the formula used for computation is a direct application of Normal probability theory. Although this mathematical model could be based on empirical evidence only, it is not coincidence that the model perfectly applies to Normally distributed statistics, and applies much less so as the statistic looks less Normal. Consider how to estimate the control limits on an X-Bar chart:

    Two parameters are calculated: the overall average and the average within subgroup standard deviation. Neither of these calculations demands that the observations be Normally distributed; however, the Normal Distribution is the only distribution perfectly described by only these two parameters.

    One parameter is tabulated: the factor (either d2 or c4) used to convert the average within subgroup variation to the expected variation of the process observations, based on the subgroup size. The estimates of the d2 or c4 factors are derived based upon the assumption of Normality of the observations.

    One parameters is defined: the number of standard deviations at which to place the control limits (usually 3). The placement of the control limits at ±3 standard deviations from the center line is appropriate only for a Normal distribution, or distributions whose shape is similar to a Normal Distribution. Other distributions may respond to this signal significantly more frequently even though the process has not changed or significantly less frequently when the process has changed. Given the intent of a control chart to minimize false alarms, this is not desirable. See Tampering .

    The Western Electric Run Tests, in fact, make use of the probability models to determine when the pattern of groups in the control chart are non-random. Without knowing that the subgroup averages should be Normally distributed on the X-bar chart, you could not apply the Western Electric Run Tests; they would have no meaning without an understanding of the probability model that is their basis.

    Similarly, the argument against using 2-sigma limits due to their impact on tampering would have little meaning without an understanding of the underlying distribution of the plotted subgroups. See Tampering .

    3. It is true that the Central Limit Theorem does not apply to the subgroup range or sigma statistics. But what does that prove? Perhaps that the distribution of the Range or Sigma is not sensitive to the assumption of Normality of the observations?

    4. Curve fitting to define Distributions, like any modeling technique, is subject to error, and statistical error is likely to be higher where there is less data, such as in tail regions of distributions. But there are techniques for dealing with this situation. See also Curve Fitting .

    What are the implications of this debate?

    1. If we use the X-bar chart, little. Both sides agree that the X-bar chart is a very useful tool, they just disagree why it is useful. As mentioned above, there would also be a question as to the validity of Run Tests in the absence of the probability model.

    2. If we use the Individual-X chart, or try to estimate process capability, we must either assume that the distribution does not matter, or fit a distribution. We can easily compare a fitted curve to the Shewhart calculations to see which best describes the process behavior. Note that the Shewhart calculations exactly coincide with the calculations for the Normal distribution, as pointed out above. See Curve Fitting .

    3. The EWMA control chart may have a couple of interesting uses, depending on your point of view:

    When we are forced to use subgroups of size one due to Rational Subgroup considerations, the EWMA chart does not require that we fit a distribution to the data. Instead, it plots exponentially-weighted moving averages, which allows the use of Normal control limits via the Central Limit Theorem. If we do not think that fitting a distribution is needed to define control limits for individual observations, then this use of the EWMA chart is not so interesting.

    A mathematical understanding of the EWMA statistic would allow proof that the EWMA control chart can be designed to be more sensitive to small process shifts. This knowledge would be useful for detecting small process shifts (shifts of approximately.5 to 1.5 sigma units) that would otherwise be lumped into “common cause variation” using the standard control limits. Note that this sensitivity is gained without an increase in false alarms (See Tampering ). Those who do not believe in the distribution as the basis for the control limits also would not accept the argument that this chart is more sensitive, or even that this chart has any valid uses. Instead, their contention would be that this chart has the possibility of promoting tampering, since it responds to “special causes” not detected through the standard Shewhart calculations.

    0
    #192752

    Steve Clapp
    Participant

    Gary, I suggested Trish use the 25% rule of thumb not for technical nor technology reasons, but simply for a change management one. I wouldn’t want my process owner to get hung up on the “look” of the control chart and miss the underlying message the control chart is telling him/her; straight-line control limits are easier for non-practitioners to digest. However, if Trish’s process owners understand why the control limits vary as sample sizes vary, then by all means she should go for the more precise tool. But I’ve learned the hard way that using all the fancy tools at your disposal doesn’t guarantee buy-in. I’ll trade a bit of accuracy for progress any day, especially in a greenfield situation like I suspect Trish is in.

    Trish, having said all of that (and, as a philosopher on this forumn is fond of saying, “it’s just my opinion”), if you are tracking every cycle time and are capturing said times in an automated fashion, and if volume is not too high, then use individuals charts (but I think your reference to ink blot tests means that you’re seeing lots of dots/inch, which means high volume). If your organization lacks the ability to easily capture cycle times, or if volume is high, then x-bar charts may be easier to generate (but you’ll lose some accuracy since you’ll be reporting on averages varying rather than individuals varying).

    Based on some of the posts I’ve read in this chain, you may also want to look into tracking cycle time within sub-processes: application-to-initial decision; initial decision-to-vendor assignment (appraisal, flood, title search); vendor turnaround; and final decisioning-to-closing. That way you will be better able to zero in on where variation occurs most.

    0
    #192755

    Trish G
    Participant

    Wow, intense discussion. I typically subscribe to Wheeler’s use of using the I Chart. I have in this instance as well. As Steve pointed out, we tend to get some volume so it gets a little thick. Not only will we measure “end to end”, but we do measures cycle times of the contributing units to enable us to pinpoint bottlenecks. The other wrinkle is that we are using a BI software called Tableau. It is very end user friendly, but its not really all that statistical in nature. I’ll be recreating a “control chart” in this environment, another reason I wanted to land on static limits. I think I will continue to use the I chart and let it be a little think on data for now. I appreciate all the discussion

    0
    #192770

    Trish G
    Participant

    Thanks Paul, I have Mini and SigmaXL so I can establish the limits using historical data, and throw them into our BI software for the process team so that they can pick up on any specical cause going forward

    0
    #192774

    Darth
    Participant

    @pkeller@qualityamerica.com Wow Paul, you sure have a lot of time to write some really long posts. Guess business is a bit slow. In any case, I appreciate the effort and thought that you put into them.

    0
    #192778

    Keller
    Participant

    I write for a living, so its not too much effort, especially since I pasted at least some of the content from my company website. The individual who asked the original question had a very legitimate concern, which had not been properly addressed by other readers. I have a passion for the topic, and like to assist where I can.

    0
    #192935

    Joan Ambrose
    Participant

    Wow! EXCELLENT discussion! Thanks to all contributors.

    0
    #192937

    Chris Seider
    Participant

    @Trish

    Be wary of using control charts of cycle times for widely different groups of data (new vs refinances).

    Also, consider my advice closely. You don’t want to wait until you have a broken down process and use the TOTAL cycle times. Find a sub cycle time (e.g. beginning to underwriter approval) as the indictator. However, I can’t tell you which sub cycle time to use. You must do some analysis and see what correlates with the total cycle time.

    Good luck.

    0
    #192938

    Trish

    Hi Chris
    Thanks for the advice, I have that covered, the process is fairly stable at this point so CC should work ok..sub process have all been identified, aligned, and being measured as well

    0
    #192947

    Trish G
    Participant

    Thanks for all the advice. There is no crying in banking. We are implementing Process Management in our organization. We are setting up dashboards that process teams can measure and monitor key measures that affect our customers and our business. We want to be able to monitor over time and detect significant changes; I was taught that control charts are a great way to do this. I am a Wheeler fan as well. I am currently exposing senior managers and process teams on the concept of control charts. They don’t seem to have a problem comprehending it so far.

    My only question was how people might have addressed considerable daily volume and the variation in it. I don’t want to “average” all of the sensitivity out of it. I have had some interesting advice from the string and it’s much appreciated.

    0
    #192948

    MBBinWI
    Participant

    @Trish – “there is no crying in banking” – unless you are the customer!

    0
    #192950

    Trish

    Ahhh, I should be more specific, no crying at Credit Unions….we take very good care of our customers…

    0
    #192952

    MBBinWI
    Participant

    @Trish – Ok, I’ll give you that. It’s been a long time since I was a credit union member.

    0
Viewing 27 posts - 1 through 27 (of 27 total)

You must be logged in to reply to this topic.