It’s an interesting problem. Ideally we’d have full requirements traceability into code so that every “place” in the code where a defect might be present traces back to a specific CTQ. In the real world of current software development practice that just doesn’t happen. What I do, and what I recommend to my clients, is to apply three rules:
1. Do not confuse black box with white box. An opportunity is a place where failure to meet a CTQ is possible. The number of opportunities for black box testing will be different from white box. With black box you don’t know what’s happening in the code so it’s valid to treat each field in a form, for example, as one opportunity. With white box, or code analysis, you’re looking for all the places in the code where a defect might be present. LOC is not a valid unit, as you realize, since there may be more than one opportunity in a single line.
2. A defect is a failure to meet a CTQ. Other “defects” may be found during reviews and testing. Document them but do not include them in DPMO. Report them as “non-CTQ issues”.
3. The ratio between opportunities and potential defects must be 1:1. Proper test coverage or code analysis often entails multiple “opportunities” to uncover the same defect. These are not the same as opportunities for the defect to occur. Do not proliferate the counts of either opportunities or defects. Every defect should relate to a specific opportunity. If you find more than one defect for the same opportunity, refer to rule 2.
Admittedly this is fuzzy measurement. But most software development is not software engineering. If it was we wouldn’t be struggling with DPMO.
.
