iSixSigma

Resolution of Ordinal Rating Scale

Six Sigma – iSixSigma Forums General Forums Tools & Templates Resolution of Ordinal Rating Scale

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • #53781

    Bradshaw
    Participant

    I have a situation where an ordinal rating system is utilized to make decisions off of visually assessed parts. The ratings are ordered by severity of appearance. I have performed an Attribute Agreement Analysis to determine the within appraiser and between appraiser agreeements. Where I am stuck is determining a resolution of the rating system. Since it is ordinal, I would expect to have some sort of estimation of resolution (i.e. ± 1 rating unit), like that which is easily obtained with a continuous measurement scale.

    Are there any tools/procedures out there to assist one in determining a “practical” resolution for an ordinal rating system gage? Thanks in advance!

    0
    #191438

    Breytenbach
    Participant

    I assume that you refer to Resolution as the smallest change a measurement system can detect in the quantity that it is measuring. For instance, a ruler’s resolution is 1 mm, a speedometer’s resolution is 1 km, and the resolution of digital display showing 2 decimal numbers is 0.01 xx (where xx is the applicable SI unit).

    I had a case where the SOP determined that a Lab had to split a composite sample into three and, after analysing them, had to report the average as an individual reading. In this case, the resolution is determined by number of repeated measurements involved: i.e., dividing by 3 will give a resolution of 0,333; dividing by 4 a resolution of 0,25.

    The common criterion for variable data is that Resolution should be at least 1/10th of the tolerance (UspecL – LspecL). No criterion exists for discrete data.

    Since ordinal measurements describe order, but not the relative size or degree of difference between the items measured, its resolution is a function of the ordinal’s definition. For instance:
    • For ordinal numbers where the ordinal scale has a constant length, such as: “very bad = 0 – 20%; bad = 21 – 40%…; very good = 80 – 100%”, the resolution would be 20%. A 5-point scale like this cannot be used with a criterion that demands Acceptable performance to be 4 or better, since the Resolution is equal to the interval of acceptable performance.
    • If the ordinal scale rank stuff in classes, such as Class A, B, C … and the scale of the classes are different, or who came first, second, or third (i.e., the results of a horse race which says who came first, second, or third but include no information about race times), the Resolution cannot be determined.

    The central tendency of ordinal numbers can only be represented by its mode (the “class” or “rank” that occurred the most times) or its median, but not its arithmetic mean. With both the Mode and the Median, the results must be interpreted by taking the scaling into consideration. See Bashkansky & Gadrich for more elaborated analysis.

    The dispersion can be expressed as a Range = the number of ordinals/classes covered by the data.

    Further reading:
    • Wikipedia http://en.wikipedia.org/wiki/Resolution, under the tag “Measurement Resolution” for other forms of resolution
    • http://www.socialresearchmethods.net/kb/measlevl.php
    • Bashkansky, E. & Gadrich, T. Some metrological aspects of ordinal measurements, http://www.springerlink.com/index/e8x73vn3m3534642.pdf presents a way to evaluate classical metrological characteristics, such as error, uncertainty and precision of single and repeated measurements based on the legitimate basic operations for ordinal data.
    Quality greetings
    wynandpb

    0
    #191444

    Paulonis
    Participant

    mbsh,

    I wonder based on your description if you are truly asking for resolution as defined by wynandpb (the typical definition) or if you are looking for something more like a confidence interval of the assessment (e.g., given an appearance rating of 6, there is a 90% confidence that the true appearance is between 5 and 7).

    If it’s more of the latter that you are looking for, you may want to look at intraclass correlations. A nice introduction is provided in When Quality is a Matter of Taste by Futrell.

    If this is the type of thing you’re after, it’s possible that the Bashkansky & Gadrich reference provided by wynandpb could have much greater detail (I’m not familar with it). You just wouldn’t find what you’re looking for under “resolution”.

    0
Viewing 3 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.