iSixSigma

“Statistical Terrorism”

Six Sigma – iSixSigma Forums General Forums General “Statistical Terrorism”

Viewing 30 posts - 1 through 30 (of 30 total)
  • Author
    Posts
  • #54048

    Philip Green
    Guest

    On page 363 of the sixth edition of his book “statistical Quality Control” Douglas C. Montgomery writes that

    ‘Kotz and Lovelace (1998) strongly recommend against the the use pf Pp and Ppk, indicating that these indices are actually a step backward in quantifying process capability. They refer to the mandated use of Pp and Ppk through quality standards or industry guidelines as “statistical terrorism” (i.e., the use or misuse of statistical methods along with threats and.or intimidation to achieve a business objective).’

    Your comments please.

    0
    #193193

    Thomas Whitney
    Guest

    Food for thought. What question is Cp or Cpk trying to answer versus Pp and Ppk? If the difference in questions is useful for improving a process capability then perhaps both are useful.

    0
    #193194

    Trish G
    Participant

    Wouldn’t any misuse of statistical methods along with threats and intimidation be “statistical terrorism”, why does it just pertain to Pp and Ppk??

    0
    #193198

    Mike Carnell
    Participant

    @Trish Everyone seems to know Occam’s Razor paraphrased to “keep it simple.” Probably not applicable but the lesser known Hanlon’s Razor is “never attribute to malice what can be adaquately explained by incompetence.” It can explain the misuse of statistics, poor book writing, selecting poor metrics, etc. Being wrong or poorly done doesn’t constitute terrorism.

    The use of the word terrorism seems more likely a couple people creating some drama (maybe they want the first Stats reality show) so they can get some attention very much like the other academician that chose to make the statement “SS kills innovation.”

    There is a ton of data out here that says things run better since we started taking these types of metrics seriously. That is business results and personally I do not consider the term “Business Results” a dirty word as it would appear K&L do.

    I will be waiting on pins and needles to see the new series “The Ppk Cell.” Maybe they can get that group that does Big Bang Theory to star in it.

    Just my opinion.

    0
    #193200

    Trish G
    Participant

    Phew…thanks Mike, I was worried I was going to have people take their shoes off before entering target setting meetings.

    0
    #193201

    MBBinWI
    Participant

    @riemannfan – Couple of grandstanders trying to make themselves out to be more than they are.

    The problem that I have is in people calling Cpk “short-term” and Ppk “long-term”. It would be more accurate to call Cpk “sub-group” capability and Ppk “overall” capability. Minitab correctly categorizes Ppk, but calls Cpk “potential” capability. This is not true, as sub-group size and frequency of sampling vs. sources of variation will have an effect on the number, none of which are the “true” potential capability although each are real calculated values. Ppk is the overall capability for the sample set, and so long as this is a random sub-set of the overall data set (and of course sized to ensure you have an adequate sample size for the confidence level desired), will provide the best representation.

    Cp is the only “true” potential capability, and is equal to Pp if pooled std dev is the method used.

    Of course, I’m not a statistician, so I might be all wet. Perhaps @joelatminitab or @robertbutler have some other comments.

    0
    #193202

    Mike Carnell
    Participant

    @Trish I have made it this long without ever needing any input from K&L or their work so I can’t imagine making dramatic lifestyle changes based something they wrote with some glittering generalization.

    I am guessing they just wanted their 15 minutes.

    Just my opinion.

    0
    #193204

    MBBinWI
    Participant

    @riemannfan – Since I don’t have the aformentioned reference readily at hand, what was the comment made by the estimable Mr. Montgomery? Since he included the quote, I hope he then made some comment regarding its veracity or foolishness. Montgomery is generally pretty level headed.

    0
    #193207

    Philip Green
    Guest

    Montgomery cites the ANSI as stating that Pp and Ppk should be used when the process is not in control. He then emphasises that “… if the process is not in control, the indices Pp and Ppk have no meaningful interpretation relative to process capability…”

    Philip

    0
    #193211

    MBBinWI
    Participant

    @riemannfan – the ANSI committee that made that recommendation clearly didn’t know what they were talking about, and Montgomery’s comment is spot on. Any system not in control cannot have useful capability statistics.

    So what other thoughts are you interested in?

    0
    #193213

    Chris Seider
    Participant

    I got raised on Cp, Cpk and like using it to compare subgroup variation versus overall variation with Pp and Ppk. However, nothing is more powerful than using the ppm total defective and use of box plots for showing various subgrouping impacts on variation.

    Except when Minitab uses Ppk to describe non-normal distribution fitting of data, Ppk to me is less useful than ppm total defective because for normal data it’s just the lower of PPU and PPL and one doesn’t know how both sides are performing. I much prefer using ppm defective, PPU, PPL, and basic graphical analysis to best understand the process’ results.

    However, as we know we often resort to visual scoreboards to assess processes across a larger process or facility so something is used. Ppk is fine but I much prefer ppm total defective since two processes can have the same Ppk but have quite differing ppm total defective.

    Cpm is also great but I’m not convinced that targets are often set appropriately.

    0
    #193214

    Philip Green
    Participant

    @MBBinWI

    I want to get an idea of how common it is for BBs and to use Pp and Ppk when a process is not in statistical control.

    Philip

    0
    #193217

    Chris Seider
    Participant

    @riemannfan

    Good luck getting an answer. It’s like asking how many people in the US make a proper stop at a stop sign before moving along. I’d say everyone knows to do it but only through sampling would you get an idea of what percentage actually does it properly.

    Tongue in cheek…..You could do a Public Service Announcement on these six sigma sites to make sure to remind people to check for process stability before usage of Pp, Ppk. :)

    @Mike-Carnell Hoped you appreciated the driving safety point. LOL

    0
    #193221

    Mike Carnell
    Participant

    @riemannfan I am probably going get all kinds of crap for this answer. You seem to be very worried about what Montgomery and K&L have to say – what does your experience say? What do you say? Not what does some book say.

    Here is my 2 cents worth. If I have to work on a problem, Capability studies and control charts are pretty much a waste of time. They don’t really answer very many questions that will get you to some type of solution. If there is a management meeting you probably crank a couple charts out to satisfy some person who is sitting there and has had some high level class so they want to look engaged and ask a couple questions like “Did you check to see if it was normally distributed?”

    Using Pp and Ppk when it isn’t in statistical control? Who gives a __X__. While you are screwing around with that type of esoteric crap I can run a few hypothesis tests and be a lot closer to a solution than you will be.

    In my business we use it as a metric (I do not terrorize myself). It is specifically the difference in the calculation that makes it interesting and valuable. I also use Cpm because I have the option of selecting to run the part weight where I choose inside the spec. I really don’t care what they think about that metric either.

    Just my opinion

    0
    #193224

    MBBinWI
    Participant

    @riemannfan / @Mike-Carnell – the problem with capability analysis is that it relies on specification limits, which all to often have no correlation with reality or anything that means anything. They are usually based on some “historical” value or pulled out of the air. History may have no bearing on today, and pulling specs out of the air is juat foolish. Thus, I can have a good capability that dissatifies lots of customers.

    I’m going to have to disagree with my good friend Mike on the use of control charts, though. You need to understand whether a process is stable or not, and control charts are just about the best way to do so.

    And I’m going to vehemently agree that Cpm is probably the most useful metric not commonly understood or utilized. It is most common that a specific target value is best, yet neither Cpk nor Ppk takes that into account – only Cpm.

    0
    #193225

    MBBinWI
    Participant

    @riemannfan – anyone doing so is ignorant of the statistics, and a fool.

    0
    #193227

    Chris Seider
    Participant

    Capability studies are often less than worthy on historical data–I’m not going to open that can of worms as to why.

    I tell (highly suggest) my clients to do short term capability studies with natural subgroups gathered and to keep the data in time ordered sequence. Of course, they’ve defined the problem and checked the MSA and done a process map and identified probable X’s beforehand.

    Of course, I get the “It’s a lot of work” look and I say “Yes”. But often afterwards I get the “wow…we just realized after collecting data across ALL the rows of the machine or across subgroup Y or looking at the time series (stability as you say) that we had issue with X” and it comes with smiles. I smile back and say “Great….glad you solved it, now find a way to control the X and not let it out of control”. To my friend, Mr. W., it would be a red or pink one. :)

    So, as my colleague Mr. MBBinWI would say, “it depends” on if a process capability study was worthwhile. Nice poking of the bee hive by Mr. C.

    0
    #193230

    Philip Green
    Participant

    @Mike-Carnell

    I’m grateful for the level of response to this topic. However, I am concerned that some of the comments are more appropriate to a discussion on snake oil than statistical quality control.

    Philip

    0
    #193231

    MBBinWI
    Participant

    @riemannfan – hmmm, you ask for comments, and got very meaningful, cogent, and on the point comments, and yes, opinions (since what you wanted comments on was somebodies opinion, this should not have been unexpected). And then you insult the participants by calling some ignorant or useless. Very rude Philip and as for me, this is the end of my participation in the thread.

    0
    #193233

    Mike Carnell
    Participant

    @riemannfan Since you like to read take a look at Juran’s Managerial breakthrough where he differentiates between Control and Breakthrough. Control is, by his definition, maintaining status quo – no change. By that definition a Control chart is a tool to maintain status quo. Breakthrough is defined as dynamic change. Most of the people that have been engaged in the breakthrough business. We don’t sell maintaining ststus quo – there is no market for someone to come into a business and make sure it continues to run exactly as it always has.

    We do sell dydnamic change/breakthrough and we can all produce the numbers to prove we do not sell snake oil so I will assume that comment was not directed at us. We all have our own way of accomplishing this but we all get it done. I have no issue have Chris or MBBinWI disagreeing with me because I respect their abilities to do the job and make a difference.

    Lets get back to the question I asked. What is your experience with this Cp, Cpk, Pp, and Ppk thing. I can’t call it an issue because it is only an issue to a few academicians. This Discussion Forum is primarily inhabited by people engaged in Continuous Improvement not perpetuating the current level of inefficiency. Now if Continuous Improvement is snake oil for you I can live with that. The one thing that I have never been able to tolerate are the people who chose to pontificate on this business. They think if they sit on the sidelines as spectators and urinate all over everything that is going on like some drunken alley cat. They think they add value. This isn’t a spectator sport.

    Just as reference point for you what I do when I consult is no different that when I am in my own factory. Consider how stupid it would be to sell snake oil to the public and then sell it to myself when I am in my factory. That doesn’t even make any sense. I have had several consultants when they are in the area walk through my factory and tell me what they would do to improve it. That is because I am always open to a different view. Something you might consider.

    Just my opinion.

    0
    #193243

    Joel
    Guest

    Philip-

    I’m siding with Seider on this one and am not particularly fond of Cp/Pp but more on that in a minute. Cp/Cpk are based only on within subgroup variation (or “potential”, although as @MBBinWI points out that’s a bit loaded since it implies the best possible but is not) while Pp/Ppk are based on overall variation. So if you want to choose which one represents the actual quality of the product, Pp/Ppk is the way to go. Using Cpk is stating what your quality qould be if there were no difference between subgroups…academics might believe that is straightforward and therefore Cpk is useful, but if that were so straightforward then you would have already eliminated those differences and Cpk would equal Ppk and you wouldn’t be asking the question.

    If you’re receiving parts from a supplier, in my opinion you’d be crazy to use Cpk instead of Ppk – what do you care how good quality would be if there were no differences between subgroups? You want to know the actual incoming quality!

    In any event, I think in the Six Sigma world people are too caught up in traditional stats like Cpk, Ppk, and really even sigma level. If your process is not of very high quality, just report a percent defective. If quality is reasonably high, use PPM. This along with a histogram showing the specs (which are hopefully good specs but that’s an entirely different topic) will allow you to commmunicate your quality level with anyone without annoying them with non-intuitive statistical terms.

    0
    #193247

    Thomas Whitney
    Guest

    Discussions on which capability measures or values to use drive me crazy if we are trying to assess capability at the beginning of a process improvement effort. Capability assessment assumes specs were set right, measurement systems to are appropriate for the granularity of spec requirements, those who set specs understand the actual process stability of the factory they are going to use and that the factory is actually stable in the long term.

    My only goal when first parachuting into a factory is to assess each of the assumptions I related above. As Carnell mentioned above how these assumptions are quantified and improved is not formulaic. I am the only person I know that does it correctly in spite of the fact that most of my competent peers seem to get processes improved by obvious luck and not perfect technique as I possess. The measures and tools and the order used I use are the correct tools and order!

    My always goal is the get a process stable first. This means to reduce within and between group variation to “zero”. My second goal is to be sure that specs desired/required make sense to whatever final stability the process can achieve. Hopefully a communication link is then made between R&D and factories such that specs respect factory stability and variation components. Lastly, I assess the capability of the process. As someone said above and I agree after years of hating it, I like Cpm.

    I think the valid use of which ever capability metric one uses ( I will give way to the notion in some instances any one of the three may be used) is to set a goal as to where the business wants their capability to rest. In my last consulting tour the company wanted a Cpm of at least 1 on all product critical characteristics. In some cases the issue was MSA, others poor within and between group variation, in others poor spec definition. All we did was insure the factory could perform to the assumptions required for a capability assessment and metric desired in the first place.

    Hope this makes sense..

    0
    #193252

    Mike Clayton
    Guest

    Mike Carnell makes most sense to me, for using data to get improvements.
    The academic community spent years studying confidence intervals for Cpk metrics, and agonizing over non-normal data (which is majority of data in many industries, which only means they need to use robust analysis methods in my opinion).

    My comments only apply to volume manufacturing, not nuclear reactor construction.

    For years many of my clients used LONG TERM SIGMA in Cpk calculations because that was what the customer was really seeing for their parts, relative to the specification limits (I agree that spec limits are often meaninglessly wide, and useless for real improvement efforts, but that is not the question here). Ppk was invented to avoid confusion over which sigma to use and academics had defined Cpk in terms of the SPC subgroup sigma (sadly). SPC has little to do with capability at all. It has everything to do with stability.

    In order to predict end customer quality from inline and end of line measurements, PPM levels required long term sigma not short term sigma, while the attempt to separate out common vs special causes in the SPC process meant short term sigma focus. These are nested efforts, one to assure stability, the other to improve capability.

    So I prefer to ALWAYS use the long term sigma for capability assessment and breakthrough improvement efforts, and the %OOC in SPC charting with many rules in place, to measure %instability, NOT some kind of very short term capability. SPC charts seldom have fully rational sub-grouping, as chart data is used for diagnostics and adjustment purposes (thus over-sampled perhaps) and not just for Shewhart-type alarms and machine action. See Box and Luceno on Statistical Process Monitoring and Adjustment. Or the many papers by Woodall in JQT on issues in SPC from past decade or two.

    I suggest ACTION as follows when % OOC is too LOW, and Cpk/Ppk is too HIGH.

    Mike’s Rules:
    A: If %OOC is zero for 3+ months, tighten SPC chart limits until you get at least 1% OOC, and use appropriate action for each rule used (more than one is mandatory).
    B: If Cpk long term (Ppk) is > 3.0 for 3+ months, start characterizing that parameter for tightening spec limits to Ppk of 2.0 max (PAT type limits). Some customers will push you to do that in any case, if parameter is really critical.

    If the SPC charts are NOT showing SOME instability (% OOC must be > zero to me, but perhaps less than 5 or 10%) then the SPC Limits are too loose…no real signal detected, and real critical parameters must give a signal to be stabilized before you can really address the long term capability. Zero %OOC is not the goal. That just mean you do not have a good sensor of process behavior.

    Maverick Lot Issues:

    If long term PPM fail rates show field failures that seem to be clustered by batch or lot number, then begin study of Dynamic PAT limits where each batch or wafer or lot of material is characterized as to its distribution, and then statistical outliers to that short term distribution are contained and studied offline for reliability-related issues. Very sophisticated “walking wounded” are often detected in this way and lot-clustured fail rates improve. See dynamic PAT methods, or outlier containment methods on web or Zero Defects programs.

    Similarly, if the Cpk long term (call it Ppk if you are Minitab or AIAG fan) is > 3.0 your specification limits are perhaps TOO WIDE and you may never get any signal useful to contain maverick batches of parts. But before you tighten spec limits, make sure your SPC limits are “signalling” and shop floor is actively working on achieving stability, not simply declaring all is well with wide SPC limits. The Japanese laugh at US/European “spec limits” and prefer targets and Cpm metrics with improvements in variability each quarter or at least each year. Just always hit TARGETS and always inprove variability.
    By focusing short term efforts on STABILITY improvement, a valid use of SPC, and long term efforts on CAPABILITY improvement for customers, a quality operating system deals with both issues, continuous improvement in variability reduction, and breakthrough improvements in product/process capability to meet customer expectations.

    If the charts never alarm, the “illusion of control” sets in for management teams.

    With no real field fails, you may still be giving customers highly variable system performance compared to competition, so paying attention to real batch to batch process distributions can pay off long term, independent of the old spec limits. Whence comes the value of Cpm (which we get free in our Cpk or Ppk reports once we understand customer targets.) And also the value of parametric databases, with traceability back 7 years or more for automotive products.
    Ford Motor Co long ago insisted that automotive suppliers of semiconductors use “Part Average Testing” or PAT methods for critical parameters, and takiung action to guard-band the test if Cpk LT was > 3.0 (since highly refined in many zero-defect projects, and quality manuals. So those final tests or inline parameters that have Cpk-lt (Ppk) > 3.0 for many months either have not really found those to be critical to customer, OR they are living in that “Illusion of Control” and will get hurt whenever any internal process parameter SHIFTS but all parts are IN SPEC, and the end customer is blind-sided, having fine-tuned their systems for the “expected historical distribution” not for the barn-door step window. We have noticed sudden DROPS in those high Cpk’s when success requires expansion, and unless we were paying attention to sources of variability all along, that increase due to new equipment or gages or locations can blindside us and hurt the customer.

    No easy answers.
    But textbooks should include engineering and economic reality issues, not just statistical purity issues. Only when data is sparse and consequences of action and no action are BOTH costly should the data be tortured a little with transformations, etc to get “closer to the “truth” and I have not seen many cases like that in my line of work. If data is that sparse, as in making only one rocket for the moon, then physical modeling and sub-component reliability testing beats any SPC or CpK assessments…until you get many rockets into production…and have deep knowledge of the underlying physics and the historical limits of that products behavior. So for example, if you have NEVER flown those rockets at freezing temperatures, you might want to think twice about using normal rubber 0-rings before testing at those temperatures, as Feynman famously showed with his ice water demo (see U tube). But with lots of data, from automated gages and volume production, there is no excuse for not using intelligently.

    0
    #193262

    Chris Seider
    Participant

    @joelatminitab
    Always good to read your perspective.
    @twhitney99
    I wanted casual readers to NOT misinterpret your comments…they must be urged to know you had your tongue stuck in your cheek.
    @mclayton200
    Classy treatise

    Generally, I don’t understand this “thinking” that specs are at the leisure of a facility or supplier to determine. Maybe it’s the industries I’ve worked in or supported, but I rarely see companies setting their own specs–even food companies have SOME set by government regulations. A supplier may SUGGEST some based on process capability but a customer worth it’s salt might have an idea of what variability it can accept. Remember, we teach specs are voice of the customer and control limits are voice of the process.

    I would never suggest not working on variability reduction but the idea of a process having a Ppk of 3+ should have the specs reduced without urging of the customer seems folly to me. If you have NO processes with rework and have way too much overcapacity, then maybe you could work on reducing variability of a process already matching customer’s needs. However, look at the back office functions for opportunities.

    0
    #193265

    Thomas Whitney
    Guest

    @cseider Thanks, I forget the entire community may not know my dry sense of humor. My comments as to me being the only right way are tongue-in-cheek.

    0
    #193267

    Chris Seider
    Participant

    @mclayton200 Yes, I laugh when people ignore or weren’t told which s.d. to use in Ppk. It’s a language and we should know to use it. I also am quite wary when people quote sigma levels because I often find people will use short term data and then still add a modifier like 1.5 and state the “sigma level”.

    It is amazing how many folks out there don’t know sigma level was DEFINED as using short term data. People then argue that it’s better to get long term data and then shift….we won’t open the can of worms of how much to adjust sigma level from long to short term. Those who advocate you MUST use long term data are the ones who sit in corporate offices and don’t have to drive improvements within weeks to satisfy their customers who are screaming. I can’t wait to do long term studies before action….short term studies and use of historical if it exists to drive process improvement.

    0
    #193272

    MBBinWI
    Participant

    @mclayton200 – spec values should never be determined based on the process. Specs always come from the customer. When you begin to “set” specs based on process variation, you have lost all credibility in your capability analysis. By its very nature, capability is what we can do compared to what we need. The mistake in most organizations is not having a clear linkage of specs to requirements (either consumer, derived from the consumer, or as required by other parts of the system).

    @twhitney99 – I was willing to agree that you’re the only one doing it right (and glad to see you’ve come over to the dark side of Cpm!).

    0
    #193276

    Mike Clayton
    Guest

    @MBBinWI:
    Sorry, but spec limits are VERY OFTEN set using process data:
    A. When the original customer specs were set very very wide, and the design proves to be very very capable for one set of process tools, and thus the customer gets used to that distribution, and tunes his application to take advantage of that, and ASKS the supplier to use PAT limits to assure that they will contain any “maverick lots” that show extremely wide range or shifted data from the expected distribution. This is VERY common in multi-year automotive contracts, where the customer (Ford, GM in two cases I remember) were never given ANY samples that were 15 sigma from the current mean, and yet their original specs permitted that kind of shift. After a year or so, the car maker having simulated a design improvement based on ALL the past data from the vendor, went ahead with their new anti-lock braking design, and meanwhile the supplier “improved” their process (reduced variability, but shifted the mean, within the old spec limits, but by a large amount, and accidents happened. As a result of that and similar examples, car makers added many many rules, and also added the requirement of PAT (part average testing) limits on critical parameters when any SHIFT beyond the 6 sigma limits of the historical distribution would be HELD for APPROVAL by the car maker (thus avoiding a price negotiation for a tightening of the old spec limits) and for the suppliers these PAT limits became the new customer requirements..gradually.

    B. Another situation is when new products are created with very wide initial spec limits, and after many lots are made, the test specification limits for parameters that MAY impact reliability or performance are set at plus or minus 6 or 8 sigma, to avoid risk of any outlier parts never modeled fully or subjected to qual testing, might be shipped. Customers in consumer market are often impacted by large unnecessary shifts (caused by new equipment or new plant sites) since they do not require re-qualifications as risk-adverse automotive customer require. Its must good business to protect some customers from huge shifts in key parameters just because they do not specify tighter limits.

    C. Data sheet limits are often set tighter than indicated by successful qual univariate corner testing, due to suspected multivariate effects that have not yet been modeled or fully tested in DOE’s, so the factory distributions are used as a guide, in absence of customer specification limits.

    D. Every industry has exceptions to the “never never” rules of academics and experts from other industries. Risk management is often based on historical fiasco’s common to that industry. The academics console themselves by naming such “aberations” as “engineering limits” rather than “process” or “customer” limits.

    0
    #193277

    Mike Clayton
    Guest

    VOC and VOP issues:
    The voices of the customer are many, not just initial specification limits as defined in an old datasheet, but issues learned during early period of their application growth, and the many re-designed of that application, triggering feedback to supply chain on impacts likely to their chosen test coverage or datasheet limits. And often the application modeling is imperfect, as it the vendor’s design modeling, so shit happens and a new customer voice is heard.

    Same story for Voice of the Process.
    Univariate limits for inline tool checks (process input and product output variables) are often set, then later tightened based on NEWLY DISCOVERED INTERACTION effects, a new voice of the process. But since VOC and VOP are stated in quality systems documents, and may require financial impact analysis, etc. there may be a long lag before risk-based containment specs are fine tuned after many months of process improvement effort.

    So these are very dynamic concepts, and reality often negates earlier DMAIC project results. Not a problem unless customer or profitabilty take big hits with the discovery and corrective action proves inadequate and a redesign is required. Design of products and processes for Six Sigma require re-characterizations over time, and these two primary voices change very often over time, even in face of long term supply chain contracts. Nobody is happy about these issues, but full multivariate characterization of complex systems designs are often not feasible.

    0
    #193283

    MBBinWI
    Participant

    @mclayton200 – Mike: Your point A supports, instead of refutes, my point. The customer always sets the specs. It is when we lose that perspective, that we begin to lose our focus on “quality” and begin gaming the numbers.

    No time to continue this discussion now. Perhaps later.

    0
Viewing 30 posts - 1 through 30 (of 30 total)

You must be logged in to reply to this topic.