THURSDAY, MAY 24, 2018
Font Size
Topic Attribute Agreement – Multiple Cases, Multiple Scorers

Attribute Agreement – Multiple Cases, Multiple Scorers

Home Forums General Forums Tools & Templates Attribute Agreement – Multiple Cases, Multiple Scorers

This topic contains 1 reply, has 2 voices, and was last updated by  Robert Butler 3 weeks, 4 days ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #705042 Reply

    Any tips as to how I can use an attribute agreement study for my data?

    Let’s say:

    I have 500 cases.
    Each case has 10 questions scored on a 1-10 ranking (so I’m assuming it’s ordinal data).
    Each case has 5 people scoring it.

    The 5 scorers may be completely different people to one another. At this point I’m looking for overall attributes across the sample, rather than how individuals are performing.

    I can’t seem to lay it out such a way that gets me anything other than error messages.

    #705063 Reply

    Based on your description it doesn’t sound like you have data that could be used in an agreement analysis (I’m assuming when you say “agreement” you want to look at something like the Kappa statistic).

    For an agreement study you have to have some kind of contrast on the same things you are measuring. For example, if you want to know how well an individual rater agrees with themselves the usual procedure is to provide a series of things (let’s use x-ray images of joint injury as an example) and have the individual rate each of the images on whatever scale is being used to assess the images. The images are given to the rater in a random order and then, at some later date they are re-randomized and given to the same rater. The rating scores are matched by images and the Kappa statistic is calculated on the before and after ratings.

    On the other hand, if by “agreement” you mean equivalence then you would use the t-test and run the Schuirmann’s TOST equivalence test. However, the problem with this approach is the same as the one for the Kappa – you need some kind of pairing of measured responses.

    Rather than continue with a series of “if” statements I think you will need to provide more detail with respect to what it is that you are trying to do.

    Some issues to address:

    500 cases – what are these – all the same kind of thing, groups of similar things but not all the same, etc.?

    10 questions on an ordinal scale – same questions for all – all questions different or do they group into question types (i.e. questions 1-3 focus on patient demographics, questions 4 and 5 are focused on economic status, etc.)

    Why 5 people per case?

    Are the cases randomly assigned?

    What do you mean by overall attributes across the sample?

Viewing 2 posts - 1 through 2 (of 2 total)

Register Now

  • Stop this in-your-face notice
  • Reserve your username
  • Follow people you like, learn from
  • Extend your profile
  • Gain reputation for your contributions
  • No annoying captchas across site
And much more! C'mon, register now.

Reply To: Attribute Agreement – Multiple Cases, Multiple Scorers
Your information:






5S and Lean eBooks

Six Sigma Online Certification: White, Yellow, Green and Black Belt

Six Sigma Statistical and Graphical Analysis with SigmaXL
Six Sigma Online Certification: White, Yellow, Green and Black Belt
Lean and Six Sigma Project Examples
GAGEpack for Quality Assurance
Find the Perfect Six Sigma Job

Login Form