iSixSigma

(Partially) Nested Design Question

Six Sigma – iSixSigma Forums Old Forums General (Partially) Nested Design Question

Viewing 5 posts - 1 through 5 (of 5 total)
  • Author
    Posts
  • #48949

    Ryan
    Member

    Hello,
    I’m trying to figure out how to understand/run what I think is a partially nested design.
    Let’s say you have a treatment that you believe is effective at reducing depression. You randomly assigned participants to either one of two conditions, treatment or placebo. You plan on testing if your treatment is effective at five different hospitals. There will be one therapist per hospital who will be providing both the intervention condition and control condition. (I consider therapist and site to be the same variable since there is one therapist per site.) The person will be assessed on level of depression/receive therapy or placebo every week for one month. At first I thought I was dealing with a straightforward nested model, where you have participants nested within therapist/site nested within treatment condition, but that is not correct because therapist is not only providing one level of the treatment condition.
    Each participant is seeing only one therapist the entire time, so if one therapist is more effective than another, one might want to control that effect. But every therapist is providing both treatments, so where does treatment effect fall into this design? Since the treatment variable crosses with therapist, is the treatment variable a separate main effect?
    The bottom line is that I would love to know what the model would look like.
    Notation: P = Participant, T = Treatment, Th = Therapist, Wk = Weeks
     e.g. P/Th = Participant nested within level of Therapist
    Any help would be greatly appreciated,
    Ryan

    0
    #166330

    Robert Butler
    Participant

     You won’t be able to separate therapist efficiency from treatment and you wouldn’t be able to separate therapist efficiency from treatment if you had two therapists per site and each was assigned only one of the treatments. 
      To separate the treatment from the therapist efficiency you would need two therapists per site and you would need to have each one performing both treatments on patients who were randomized to therapist and to treatment within a hospital.
      With the situation you have, if patients are randomized to therapy within each hospital you could test for the effects of treatment/therapist efficiency and hospital/therapist and you could look at the treatment/therapist efficiency x hospital/therapist interaction. You could then build custom contrast statements to allow a comparison of each treatment/therapist across hospital/therapist. These custom contrasts would provide some kind of a test of therapist efficiency by assuming across hospital/therapist there should be no significant difference among placebo or among treatment
      You have the additional problem that you are running this over time so you have repeated measures on each of the patients.  Your model would also include a time effect but you will have to have software that can handle repeated measures and you will have to know how to code the data so that the program will recognize that the measures are repeated and are not independent.
      None of the above addresses the problems of patients populations available to each of the five hospitals.  For most medical issues things like patient gender/age/race/economic background etc. will have an effect.  If the five hospitals all draw from a similar patient population this won’t be a problem but if there are significant differences in the patient pool available to the hospitals then the confounding effect isn’t limited to hospital/therapist it now includes patient demographics and the differences could be extreme enough to call into question any claims you might make with respect to treatment efficacy.

    0
    #166331

    Ryan
    Member

    Robert,
    Thank you for your thoughtful and detailed response.
    The study I am proposing would involve a therapist providing both treatments. I can see how hospital effect could not be separated from therapist effect if there is only one therapist per hospital, but I do not understand why I cannot separate therapist effect from treatment effect. Perhaps I shouldn’t use the term “separate.” Couldn’t I use an interaction term (as you said) therapistXtreatment to see if the effect that treatment has on a variable of interest (e.g. depression) depends upon the therapist? If I find that there is no significant interaction effect, then that bolsters the notion that it is unlikely that the treatment effect on depression depended on which therapist provided the treatment. 
    After reading your response and thinking about this further, I do not think I’m dealing with a nested model at the level of therapist and treatment, since therapist and treatment cross each other at all levels.
    Perhaps the model would look like this:
    depression =  treatment     therapist     week     treatmentXweek treatmentXtherapist       treatmentXtherapistXweek
    You are right that this model would not explain cohort differences. I suppose before running the full model, I could assess potential differences in (1) proportions in gender, (2) means of age, etc. between hospitals. If there are any differences, what would I do? I certainly can’t use them as covariates… Perhaps a nested variable where, for instance, gender is nested in hospital (aka therapist) –> Therapist(Gender).
    I’d love to hear your thoughts if you have the time.
    Ryan

    0
    #166338

    Erik L
    Participant

    Ryan,
     
    Looking at the Factor Relationship Diagram (FRD) for this scenario makes me question how you are going to tease out any interaction effects amongst the factors in the design.  The experimental unit is the patient’s depression level (on average) based on the random assignment to either the ‘treatment’ or the ‘placebo’. 
     
    If I understand the current protocol, there is not a scenario where the same patient is exposed to both conditions across the study.  Since this is lacking, there is not a crossed relationship which would eliminate the ability to assess interaction effects amongst that level of the design. 
     
    Could the protocol be modified to randomly assign the participants/hospital amongst the treatment level for two weeks total time and then rotate them to the other setting (in a blind manner to the physician and patient)?  Are you concerned about some residual effects of the treatment which might bias the results if they go treatment-placebo?
     
    If there is a larger # of therapists which might prescribe this treatment (if efficacy is demonstrated) have you thought about modeling the therapist as a random effect?  Around the question that you had relative to demographic factors, and if there appears to be issues that might bias the results, those terms could be included in the model.  Let’s say we believe race is significant.  The major strata of race could be modeled through an indicator variable for the factor and there could also be the addition of weighting terms to take into account known % differences in the factor.  Hope that has helped with some questions that you may have had.  It looks like it’ll be an interesting study.
     
    Regards,
    ErikThis thread has been moved to the Healthcare discussion forum. Please click here to continue the discussion.

    0
    #166341

    Robert Butler
    Participant

    You are correct.  I was thinking in terms of splitting out the effects of hospital, therapist, and treatment.  If you are going to call hospital and therapist the same thing and if the patients are randomized to treatment (we’ll ignore patient population variables for the moment) then you can build a model where the terms would be
    Y = fn (treatment, therapist, treatment X therapist, and time)
     The point of looking at the interaction being as noted – to see if there is an overall interaction between the two which would suggest differences in treatment or differences in placebo due to therapist. 
     The usual practice is to assume if the interaction isn’t significant there isn’t a problem.  There are others, however, who will insist on building the contrasts I mentioned in the prior post to test for treatment/therapist differences even if the interaction isn’t significant. 
       I have a mixed reaction to this approach since it smacks of a snark hunt for significant differences.  If the significance of the interaction term is “close” to significant (between .1 and .05) and if prior to the start of the effort, it was clearly stated that the investigator believed there would be a therapist effect on the kind of treatment offered and it was stated which therapist was believed to be different from which, I’ll run the contrasts just to see if any of the results of the effort support these beliefs.  If they do, I’ll note this fact and I’ll also note the insignificance of the interaction term. I will use the data to do a power analysis for a follow on study should the investigator wish to check this issue further.
    As for the patient issues – If there are significant differences from hospital to hospital then you have a problem because you are going to have a confounding of hospital/patient population/therapist. If some of the hospitals have similar patient populations then you could examine the subset of the data for which this is true and build a model like the one above but with the possible addition of patient variables.

    0
Viewing 5 posts - 1 through 5 (of 5 total)

The forum ‘General’ is closed to new topics and replies.