How Do I Select the Right DoE Strategy with Different Supplier Material Qualities?
Six Sigma – iSixSigma › Forums › General Forums › Methodology › How Do I Select the Right DoE Strategy with Different Supplier Material Qualities?
 This topic has 14 replies, 2 voices, and was last updated 2 years, 2 months ago by Robert Butler.

AuthorPosts

February 25, 2019 at 9:20 am #236693
Alessandro SilvestroParticipant@alek83 Include @alek83 in your post and this person will
be notified via email.Dear All,
I am struggling to find the right DoE Strategy for the following.
Problem:
we have a process that is capable of working only one high quality material. This generates global outsourcing issues and potential collapse in case our single supplier decides to step out of our small (for them) business.Objective:
Understand the key factors responsible for knives bolster quality and be able to produce the same bolster quality with different materials and suppliers (process standardization)DoE Strategy:
we have now in stock 3 additional materials which range anywhere between these values:
Tensile Strength: from 600 to 800 N/mm^2
Elongation: from 20% to 10%
Roughness: from 2 to 0,5 µmWe want to have a sigma forging process that can achieve good results, independently of the material quality within above ranges. We have identified the following process parameters:
Energy Flow (Ampere)
Upsetting speed (mm/secs)
Clamping pressure (Bars)
Anvil speed (mm/secs)
Upsetting delay (secs)I lean towards a 2^(51) factorial design with center points for each supplier but this would lead to individual process “recipes”. Much better would be to have one robust process that can work with any material having properties falling in those ranges.
Which kind of DoE would you recommend?
thank you very much for all your help and expertise,
Alessandro
0February 25, 2019 at 11:57 am #236698
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.Since you are concerned with the ranges of incoming material properties in addition to the things you vary in your process a better approach would be to run a saturated design on the 8 variables
Tensile Strength: from 600 to 800 N/mm^2
Elongation: from 20% to 10%
Roughness: from 2 to 0,5 µm
Energy Flow (Ampere)
Upsetting speed (mm/secs)
Clamping pressure (Bars)
Anvil speed (mm/secs)
Upsetting delay (secs)You would select random material from all three suppliers to fill in the low and high values for tensile strength, elongation, and roughness and identify your high and low settings for your in house process parameters in the usual way. If you have overlap with respect to supplier raw material quality you could run a couple of replicate points using alternate material. You could run a main effects check in 18 experiments if you just wanted to run a semisaturated 2 level factorial design or, if you have the computing firepower, you could plug your data matrix into a Doptimal design generator and probably identify a design of about 12 runs.
0February 26, 2019 at 4:31 am #236714
Alessandro SilvestroParticipant@alek83 Include @alek83 in your post and this person will
be notified via email.Dear Robert,
thank you very much for your prompt response, I really appreciate it! I am not sure however I understand what you propose.
It is extremely time consuming to get in our lab tensile strength, elongation and roughness measured.
All we can do is trust our supplier nominal material property specification sheet.Our intended function is the bolster volume to surface ratio, attached you may find a picture.
We are pretty sure that higher tensile strength and smaller roughness would enhance the process, so I would expect the saturated design to confirm this. But what shall I do later?
I need a robust process (and DoE strategy) that minimizes material requirements (800 N/mm^2 and 0,5 µm is great but kicks out most of the potential supplier base).
thank You, kind regards,
Alessandro
0February 26, 2019 at 12:36 pm #236719
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.I think we need to back up a bit. If I understand your most recent post correctly you get a spec sheet from a supplier which provides tensile strength, elongation, and roughness measurements on each shipped lot. If this is the case then this would argue that you have zero control with respect to independently varying these three measures – for any given lot they are what they are.
If this is the case and if you want to check your hypothesis that higher tensile strength and smaller roughness would improve the process then, you will have to lot select for four different combinations of tensile strength and roughness. Assuming you can do this what you will need to do is use the methods of computer generated designs (DOptimal is one that is readily available) to build a design to test this. Of course, if you want to toss elongation into the mix then you would have to lot select on this variable as well to come up with the four combinations of the three variables (1, 1, 1), (1,1,1), (1,1,1) and (1,1,1). Given you can do this you would build a full 2**8 matrix, restrict the combinations for tensile strength, elongation, and roughness to the 4 combination and use Doptimal to find the minimum number of experiments necessary to test all 8 items.
The above is, of course, based on a lot of assumptions. We are assuming the spec sheet really reflects measurements on the provided lot of material and isn’t some generic Good Housekeeping Seal of Approval. We are assuming you have the ability to do lot selection/retention. We are assuming that, regardless of what the manufacturer is sending you, it is possible for tensile strength, elongation and roughness to vary independently of one another.
If you could provide more detail on the above assumptions we’ll see if we can come up with some suggested courses of action.
0February 27, 2019 at 9:17 am #236733
Alessandro SilvestroParticipant@alek83 Include @alek83 in your post and this person will
be notified via email.I will try my best to be clearer on objective and assumptions:
1) as you mentioned, we have zero control on the supplier material properties but we know they fall within the ranges above
2) due to this variability, we do not want to have material properties as control factor. This means we do not need to talk about elongation, roughness and tensile strength because we do not control them.
3) we have 5 process parameters that we do control. Should we add supplier as factor with 3 levels (i.e. 3 different suppliers)? I found this Statgraphics video where it is explained at 4:27 how to declare 3 suppliers for a Doptimal generated design. Could it be the right approach? If so, why supplier is declared as “controllable” factor and not “noise” factor?
4) Another option is to run a DoE for each supplier, 3x DoE in total for 3x Suppliers. Not the best but I think still doable.
5) we need a process window large enough that allows us to manufacture different supplier qualities, regardless of their variability within their material properties. (always within the ranges mentioned in the previous post). This robust process needs to compensate material weakness or achieve optimality independently from material variability.
6) we want to qualify as many suppliers as possible.
Hope it helps and thanks for your patience, really appreciate your help!
all the best,
Alessandro
0February 27, 2019 at 2:52 pm #236736
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.Since each of the supplier lots are allowed to vary within a range of (separate?) values for tensile strength, elongation, and roughness and since you only want to run an analysis where you are controlling your in house process parameters the best thing to do would be to review your collection of spec sheets from prior lots of raw material and summarize the reported property values. This will give you a picture of the distributions of tensile strength, elongation, and roughness you have accepted in the past. Once you know these distributions you should look over the spec sheets of those material lots which you do have on hand to see where their properties fall relative to prior history.
If, collectively, the current raw material lots have levels of the three properties which span say 95% of the range of each property of interest you could take that data, ignore the lot source, and attach a complete design matrix for the in house properties to each of the lots and use this matrix of design points as a basis for identifying runs in a DOptimal design. (If the current supply of raw material does not meet this requirement you will have to put aside some portion of the current lots and wait for your suppliers to send you material that, when combined with your saved samples, does cover an acceptable range of the three variables of interest).
What you will have will be a design that will allow you to check for the impact of your in house variables as well as the impact of the material property variables. What you will get from the analysis will be a predictive equation involving both the in house variables as well as the material property variables. The point of the predictive equation is NOT to try to control material properties, rather it is to give you and understanding of how well your in house variables can be expected to control your process given that you are NOT controlling material lot properties.
What you would do with this predictive equation is take the prior measurements from the old spec sheets for material you have already used, set you target and tolerance for the measured response and then use the equation to identify settings for each of your in house variables that would result in a within tolerance predicted response. The matrix of values for the in house variables used in the equation would be based on the range of settings you know you can run for each in house variable.
The matrix of predicted responses will tell you what you need to know with respect to your ability to control product properties given that lot properties will vary and how much control you can or cannot expect given the properties of the incoming raw material.
Based on your description I don’t think a basic design run on a lot of material from each of the three suppliers would be of much value. If the available lots from these three suppliers just happen to have about the same values with respect to tensile strength, elongation, and roughness then all you will have done is essentially replicate a saturated design 3 times.
As for treating the suppliers as a noise variable – you can’t. If you are going to try to run a Taguchi design what you need to remember is that a noise variable is a variable you would rather not have to control when running your process. However, when running the design you HAVE to control the variable you have designated as a noise variable because you have to be able to set it to high and low levels in order to meet the design requirements. Based on what you have said, you do not have this level of control.
0February 28, 2019 at 2:32 pm #236750
Alessandro SilvestroParticipant@alek83 Include @alek83 in your post and this person will
be notified via email.we are checking the spec sheet we have just received but let’s assume:
5x process parameters as factors with 2 or 3 levels each.
supplier 1 670 Rm and 19% elongation
supplier 2 640 Rm and 15% elongation
supplier 3 650 Rm and 19% elongationwhat are here the levels and the factors to specify for a Doptimal design? Supplier as a factor with three categorical levels? Because material properties as factors cannot be changed and are constant.
Response is the surface to volume ratio measured after the process runs.
I have never done this before in minitab 17 but I have also downloaded today a trial version of JMP.
thanks a million!
0February 28, 2019 at 5:26 pm #236768
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.Ok, let’s make that assumption. You have the following:
supplier 1 670 Rm and 19% elongation
supplier 2 640 Rm and 15% elongation
supplier 3 650 Rm and 19% elongationTherefore what you have are the following combinations for the two factors Rm and elongation
Supplier 1 +1 and +1
Supplier 2 1 and 1
Supplier 3 1 and +1The key here is to remember that it is rare, even when you have the capability to actually control the settings of a given variable, to be able to get back to the exact same setting for the “low” or the “high” value. As long as the range of values for some “low” setting do not overlap the range for some “high” setting you can view the values in the manner indicated.
So what you now have to do is build a matrix of experiments that the Doptimal design can use for construction. For everything at two levels this would be the full 2**7 matrix (I’m saying 2**7 because, using the above example, you only have two supplier variables in addition to your 5 process variables).
If we assume all you have are the three choices for the suppliers listed above you would instruct the Doptimal program to only consider those experimental combinations that have those three combinations for the variables Rm and elongation and have it build a design that will allow an examination of all 7 variables based on the restricted subset. Once you have the results of the design you would run the usual regression analysis and use backward elimination and forward selection with replacement (stepwise) regression methods to construct your reduced model (you would want to run the analysis both ways because DOptimal designs are rarely perfectly orthogonal).
Let’s assume the residual analysis indicates all is well with the final model and let’s assume the final model has both supplier terms and process terms. Given that model you can fix the levels of the supplier values at whatever you choose and then run optimization studies on the process variables. This analysis will tell you what you can expect given that you have no control over supplier variables.
In light of the statement you made in your second post – “We are pretty sure that higher tensile strength and smaller roughness would enhance the process” I think you should be prepared for the possibility that your ability to influence product quality using only in house variables will be limited. However, if this turns out to be the case, because of your efforts you will have real proof that approaching the problem just by looking at in house variables is not enough.
0February 28, 2019 at 9:22 pm #236778
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.Addendum to the last post: One thing to keep in mind when your low’s and high’s have ranges of values is the “fuzziness” of the 1 and 1 settings can be an issue. If the highest low value and the lowest high value for a given variable are too close together you may find you will lose information with respect to some variable interactions. In your case this would most likely apply to any attempt to check the interactions of the supplier variables with one another or with the in house variables.
The way you check to see if this is a problem requires scaling the actual low and high values to their actual negative and positive settings and not pretend that any low setting is automatically a 1 and any high setting is automatically a +1. The way you do this is to take the max and the min values for a given variable and scale the entire range of that variable from 1 to 1.
To do this compute A = (Max value + min value)/2 and B = (Max value – min value)/2 and scale each setting according to the formula
Scaled X = (Actual X – A)/Bonce you have the actual matrix of X’s expressed as levels ranging between 1 to 1, use this matrix to compute the scaled values for whatever terms you expect to be able to check with the design you have built and then check the resulting matrix to see if you get confounding.
This check is performed by assigning a simple count variable to each experiment in the design (1 for the first experiment, 2 for the second experiment, etc.) Regress the counts against the model terms as expressed in scaled form and look at the VIF’s for each of the model terms. Those with a VIF > 10 are too highly confounded with other terms and cannot be estimated.
An even better way to do this is to look at the eigenvalues/condition indices of each of the model terms but to the best of my knowledge this capability is not present in most computer packages.
0March 1, 2019 at 7:51 am #236784
Alessandro SilvestroParticipant@alek83 Include @alek83 in your post and this person will
be notified via email.thank you very much!
after reading your suggestions and consulting with my team, I have built the attached plan, please have a look.
1) Does this DOptimal plan make sense to you? is it in Minitab doable or would you suggest other packages?
2) If for example supplier 2 and 3 have both +1 as tensile strength level, how does any software understand which one is what? Conversely, how do I make sure I can test both supplier at +1?
3) The last 3 material properties were recommended to me based on expert judgement and therefore declared as categorical assessment (1 good, 1 bad). Measure large sheet metal areas to conclude that roughness, homogeneity and conductivity is the same everywhere we need it to be is basically not feasible and affordable.
Would you keep them or rather kick them away at this point?I hope I am not bothering you too much and that you enjoy this discussion as much as I do :)
have a great day,
thanks, regards,
Alessandro
Attachments:
 DOptimalPlan.xlsx
You must be signed in to download files.
0March 1, 2019 at 10:37 am #236786
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.I’m a statistician – discussions like this are never a bother. :) As to your plan – there is a problem with the way you have laid out the supplier information – specifically it does not match the example you provided earlier and which I used as a starting point for my post yesterday.
In the earlier post you gave the following:
supplier 1 670 Rm and 19% elongation
supplier 2 640 Rm and 15% elongation
supplier 3 650 Rm and 19% elongation…and I translated the above settings into the following design matrix for the two variables RM and Elongation
Source…..RM…. Elongation
Supplier 1 +1 and +1 ……..Thus the combination is (1,1)
Supplier 2 1 and 1 ……..Thus the combination is (1,1)
Supplier 3 1 and +1 ……..Thus the combination is (1,1)What you have done now is two things
1. You increased the number of supplier variables from 2 to 5.
2. You are trying to use just one lot of material from each supplier to fill out the supplier part of the design matrix.If you choose to go with 5 supplier variables then, at a minimum and assuming two levels per variable (I just checked with a Doptimal run) you will need to have material from your various suppliers which meets the following conditions WITHIN a single lot of material from a supplier:
Code: (Level Variable 1, Level Variable 2, Level Variable 3, Level Variable 4, Level Variable 5)
Combination 1 (1,1,1,1,1)
Combination 2 (1, 1,1,1,1)
Combination 3 (1, 1, 1, 1, 1)
Combination 4 (1, 1, 1, 1, 1)
Combination 5 (1,1,1,1,1)
Combination 6 (1,1,1,1,1)As I mentioned in the last post – since you have to take what is sent you won’t hit the same low and the same high values for each variable across the combinations – the issue is to make sure that within a given variable the range of values you are calling “low” do not cross over into the range of values you call “high”.
It also now appears you do not have a physical measure for roughness, homogeneity, and conductivity so you now have to treat the measures as a two level rating. This is fine but do not treat the ratings as categorical when you build the design. The ratings do have a rank ordering and, as long as your in house expert is confident he/she can discriminate between good and bad, you can treat those variables in the same way you would treat the other variables when building your design.
The really big issue would be identifying lots of raw material from the various suppliers whose spec sheets in combination with your in house ratings, could populate the supplier design matrix I listed above. In my experience it is almost impossible to build a matrix for more than 3 variables at two levels given this method of variable level selection and control (actually lack of control). I’m not saying you can’t do this but you should review your prior spec sheets and your prior in house ratings for past lots of materials across all suppliers just to see if, in the historical data, you have a subset of measures that would have met the requirements of the matrix.
As for telling the difference between suppliers my interpretation of your problem was that you wanted to be able to optimize the process regardless of supplier. If you are able to populate the supplier matrix with material across suppliers then the resulting model would describe the behavior of your process with respect to variables only. Thus, when you received a new lot of material from any supplier you could plug the corresponding values into your predictive equation and then ask the equation to identify your in house process parameter levels that would result in optimum product.
0March 1, 2019 at 2:55 pm #236789
Alessandro SilvestroParticipant@alek83 Include @alek83 in your post and this person will
be notified via email.Just when I’ve got the feeling to be very close to the right DoE Strategy…:D
I am an industrial engineer with crossfunctional background who happens to have factorial design and RSM experience, however I clearly lack a solid understanding of this DoE complexity.
By the way, we could connect in linkedin if you like: https://www.linkedin.com/in/alessandrosilvestro/
I do want to optimise the process regardless of the supplier but given everything discussed until this moment, either:
– I perform a DOptimal DoE with 5x factors and 2x material factors. I am still learning from you and online research how to do it right. I could also rank by roughness with discrete numbers the suppliers and try to fit in the 3rd material variable but I trust 100% your experience which does not recommend to go further than that.
– or run a 2^(51) factorial DoE on each supplier (my initial plan) and optimise each supplier differently, to the extent allowed by those material properties I won’t get represented into a predictive equation. I may need 3×16=48 runs for this but unless you have strong arguments against it (I do expect every supplier to have different meaningful factors and optimal levels), this would be my plan B to go.
thanks, regards,
Alessandro
0March 1, 2019 at 5:39 pm #236983
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.To your points
– It’s not that I don’t recommend trying more than three rather it is the odds of finding raw material lots (a minimum of 6 if we go with the Doptimal design I mentioned earlier) with the right levels for each of the 6 combinations. If you have prior data on lots of raw materials it would still be worth examining the data to see if there were lots whose overall material properties would have met those six combinations. You might get lucky and you might find your supply chain is such that you could actually run an analysis including the five supplier variables.– The biggest problem with running a 2^(51) design on a lot of material from each of the three suppliers is there is no guarantee the lot is representative of the lottolot variation one might expect from each supplier. Based on some rather bitter past experience, I think you would find the following:
1. The final equations for each of the suppliers will differ with respect to significant inhouse terms and/or significant differences in corresponding coefficients.
2. When you attempt to test the developed equation for a given supplier using a new lot of material from that supplier you will find the predicted correct settings for optimizing the process (based on the old lot) do not apply and do not provide guidance with respect to optimizing.If you choose to try running the 2^(51) with a lot of material from each supplier try to do the following:
1.Across suppliers try to pick the three lots so that two of the supplier variables of interest match the small matrix we talked about previously and let the other three fall where they may.
2. Build the three models for each of the suppliers. Make sure you build each model using scaled levels (1 to 1) for the inhouse variables.
a. Take the three models and examine them for term similarity and difference.
b. Since the models are based on scaled parameters you can compare term coefficients across models directly. This will allow you to easily see not only term similarities and differences but also see what happens to the magnitude of the coefficients for similar terms in each of the models.3. Next pool the data from all of the experimental runs. Rescale all of the inhouse variables to a 1 to 1 range. If you were able to find three lots that allowed you to check the two supplier terms add those terms (again scaled to a 1 to 1 range) to the list of parameters you can consider and rerun the model.
a. Take this model and compare it to the results of the three separate models again looking for similarities with respect to term inclusion and coefficient size.4. If you can’t find lots across suppliers that allow you to check some of the supplier variables then take the data from the three runs and assign an additional dummy variable to the data from each fractional factorial run (1,0,1 will suffice).
a. Build a model based on all of the data and including the dummy variable which will represent material lots/supplier differences.
b. Examine this equation in the same manner as above – looking at similarities/differences in model terms between this overall equation and the three separate ones.
c. If there seems to be “reasonable” agreement between this model and the three separate models (any term in the “big” model is present in at least one of the three separate models and the coefficients of those terms are of the same magnitude and direction) and IF the coefficient for the lots is such that any unit change in that variable can easily be overcome with combinations of changes in the other terms in the model then it would be worth testing the “big” model with a new lot from any one of the suppliers to see how well it can predict the outcome.
d. If, on the other hand the “big” equation has a “number” of terms (sorry, I can’t be more precise than this) not found in the three separate equations and/or there are large differences in coefficient size or direction this would argue there is more to the issue of suppliers and lots than you expected. It would still be worth checking the equation based on all of the data just to make sure.
e. The worst case is one in which the final equation based on all of the data has just a few inhouse process terms with small coefficients and a term for the lot/supplier variable with a large coefficient whose size is such that no combination of the other variables in the final model could overcome lot changes. In this case about all you can say is that lottolot variation is impacting your process in ways that cannot be overcome with the current group of inhouse variables you are using for process control.0March 6, 2019 at 11:28 am #237067
Alessandro SilvestroParticipant@alek83 Include @alek83 in your post and this person will
be notified via email.Dear Robert,
we are working on all your suggestions and as soon as we find out the exact number of experiments we can afford (internal Lab is understaffed and has also issues with the response 3D measurement), I let you know our DoE proposal, in the meanwhile hope you are having a great week and thank you very much for all your help!!!
kind regards,
Alessandro
0March 6, 2019 at 4:14 pm #237097
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.Ok, sounds good.
0  DOptimalPlan.xlsx

AuthorPosts
You must be logged in to reply to this topic.