iSixSigma

Defect or bug – a strawman

Six Sigma – iSixSigma Forums Old Forums General Defect or bug – a strawman

Viewing 19 posts - 1 through 19 (of 19 total)
  • Author
    Posts
  • #43733

    Anonymous
    Guest

    Here is a ‘srawman’ argument against the common ‘Six Sigma’ view that a software bug is the same as a defect.
    Many Six Sigma consultants are touting Six Sigma as a ‘defect reduction’ methodology, but in reality Six Sigma is a business improvement process. To my mind Six Sigma it is pretty limited as a defect reduction method anyway because it regards all defects as equal, which clearly they are not.
     
    Another mistake is to consider a software ‘bug’ as ‘defect.’ Even if we apply ‘advanced’ statistical methods  – including correlated defects – which G.E.’s version of Six Sigma ignores, a bug does not correspond to a defect. So what is a ‘bug?’ To my mind a bug is an anticipated behaviour in a software program. The reason I claim it is unanticipated is because most software development tools these days include syntax checkers, and other tools to eliminate defects.
     
    OK, so what happens when a software program ‘crashes’ at a customer site I hear to say; isn’t this an example of a defect? No, not necessariliy – because defects can only be categorized after failure analysis. Take an  example – a simple SQL query to search for someone called O’Toole, which may well lead to an unanticipated behaviour because the use of the ‘son of’ character is also used in the query to delineate the search term.
     
    Is this a defect or a bug? To some, any failure of service is a defect, but in this case the program has not failed – it worked correctly! It’s a bit like wearing the wrong colour shirt – one that ‘clashes’ with ‘the wrong trousers.’
     
    Therefore, we should not equate defects and bugs – rather we should incorporate better error traps and ‘de-bug’ in our software more effectivly. This approach has more in common with the Toyota Production System Lean than with Six Sigma.
    Andy Urquhart

    0
    #139152

    Adrian P. Smith
    Participant

    Hi Andy
    I think what you are missing here is that in Six Sigma it is the CLIENT who defines what is a defect and not the software supplier. Do you think the customer considers the failure of the SQL search to function as they expect a defect? Of course they do.
    Phrases like “in this case the program has not failed – it worked correctly!” are typical of non-Six Sigma thinking.
    Bugs and defects are not the same. A bug is an inadvertant mistake by a software programmer that may or may not be visible to the customer. A defect is undesired behaviour of a software program which is visible to the customer and which may or may not be associated with a software bug.
    Six Sigma can help in both cases: avoiding bugs by creating more robust development and testing processes, and avoiding defects by more clearly defining and understanding the customer’s requirements.
    Rgds,Adrian

    0
    #139153

    Anonymous
    Guest

    Hi Adrian,
    Thank you for participating in this debate …
    First of all I disagree that it is the client who defines what is a defect. I also disagree that ‘Six Sigma thinking’ is limited to one point of view.
    I’m glad we agree that bugs and defects are not the same; but I disagree with your restricitons on the definition of a software bug.
    I also disagree that the use of Six Sigma can offer much towards the ‘creation’of robust software development, although I’m happy to admit it has a place in ‘testing.’
    I also agree that QFD can help define all customer needs – both internal and external; but this procedure is not limited to Six Sigma.
    Cheers,
    Andy
     
     
     
     

    0
    #139184

    Stevo
    Member

    I agree with your customer view, however if a bug requires rework,  I consider it a defect.
    Stevo
     

    0
    #139185

    Anonymous
    Guest

    Thanks for participating Stevo – I often read your words of wisdom.
    Let me give another example. Take my mobile phone for example: when I first tried to check my messages, unlike previous phones, I obtained an error  message from the provider, but try as I might I could not retrieve my messages.
    Now it occured to me at the time whether this was a defect or a bug. The qustion was important to me because contractually if it is a defect I could return the phone.
    Of course the answer I got is it is a bug and the reason is because the programmer did not anticipate what I might do in order to retrieve my messages.
    My counter argument was it is entirely reasonable to dial 121 in the UK to retrive messages, but for some reason the programmer chose a different ‘shortcut.’ But it is not documented in the book!
    Why is this distinction important?
     I think it is important because the presence of a defect implies the phone is defective – it is not. There is nothing wrong with the phone provided one knows how to use it. Perhaps one could argue the documentation is defective; but this detracts from my view the phone has a bug.
    A more important issue is what is a good strategy for eliminating bugs – and I’ve already stated my view – it is by adding more error traps and error handling, and by trying to anticipate what the user is trynig to do to avoid rework as you point out. However, if you regard the bug as a ‘defect’ then the addition of such software is ‘non-value added and wasteful, and it seemed to me to encapsulate the difference between some of ‘Six Sigma thinking’ and some of  ‘TPS thinking.
    Of course I might well be wrong and I look forward to reading the view of others.
    Cheers,
    Andy

    0
    #139188

    Stevo
    Member

    Andy,
     
    I usually don’t get involve in these serious conversations because I have neither the experience nor the education to add much value.  But I’m on cold medicine so here goes:
     
    If you look at the product itself, then I can see both sides.  However if you look at the process holistically, where does the process end?  To me, it ends with a customer knowing how to use our product well.  If our training material is not doing its job or if we haven’t Poka Yoked it well, then we still hold some of the responsibility.
     
    Thank god for the good drugs,
     
    Stevo

    0
    #139198

    OLD
    Participant

    Andy U.:
     
    I tend to agree with Adrian P. Smith’s perspective. Maybe too simplistic but what is the purpose of the software? Ideally to allow a user to do a task(s). Correct?
     
    Task = Doing what the software is designed to do, capable of doing, or, stated as being able to do.
     
    A DEFECT – keeps the user from doing the task (root cause related to a failure in design, manufacture, or application).
     
    A BUG – keeps the user from doing the task (root cause related to a failure in design or execution of the design)
     
    LACK of USER KNOWLEDGE – keeps the user from doing the task (root cause is that the user does not know how to use the software due to poor training, poor documentation, poor listening skills, etc.)
     
    As a user of the software, I don’t think I would see a difference in DEFECT and BUG (I think the distinction lies within the respective root causes). As a producer of the software, I would make the distinction between all three as it would impact how I approached solutions to eliminate the problems at their source (to your point). Thin line……
     GOOD LUCK! OLD

    0
    #139205

    McD
    Participant

    I think you want to be a little careful in throwing around terms like ‘bug’ and ‘defect’ as they relate to software.  Software types have very specific meaning for the term ‘defect’, and Six Sigma software shops count defects, most of which (hopefully) never reach the customer.
    In software development, a failure to meet requirements (which for some operations might be some internal requirement) is called an error.  An error which escapes the phase is called a defect, and a defect which escapes to the customer is called a released defect.  Managing these escape rates is an important aspect of software development, and a place where Six Sigma can be, and has been, applied with considerable success.  Avoiding the errors in the first place has tended to be somewhat less successful.
    Bug is an imprecise term which generally isn’t used by shops which are paying attention to quality.  It tends to be more in vogue among garage shops and kids playing in the basement.
    And I am absolutely at odds with your statement “I also disagree that the use of Six Sigma can offer much towards the ‘creation’of robust software development”
    Software development is a process, like any other process, and like any other process, it can be improved.  The way we improve processes is with Six Sigma.
    But it goes beyond that.  Software developers use DFSS as their process for developing robust software.  DFSS is quite similar to the traditional SDLC, but offers more robust tools at every step along the way.
    Certainly different tools are used in software development with a different frequency than in manufacturing, just as different tools tend to be used in any transactional process.
    For example a gage R&R is often impossible to perform for many transactional measurements. Get over it, move on.  On the other hand, tools like ABC/BOC or logistic regression tend to be used more frequently.  But the process is the same.
    I might even go so far as to argue that the opportunities are not nearly as rampant in ‘testing’ as they are in development.  The money lying around in development waiting to be picked up is absolutely amazing.
    –McD
     
     

    0
    #139208

    Anonymous
    Guest

    The rational I used to determine whether there is a defect or not is the system/software/process is defective. In other words, if there is a system crash.
    If we can’t claim the software is defective then any unexpected behaviour must be a bug.
    Cheers,
    Andy

    0
    #139210

    Anonymous
    Guest

    Most software developers in the Cambridge are still use the term ‘bug,’ and in the context I mentioned.
    Furthermore, it doesn’t surprise me that many Six Sigma ‘bible punchers’ still do not understand the principles of robust design.
    My comments had nothing to do with software development processes, but sine you raised it I’m not aware of any in the Cambridge area using the DFSS process IDOV.
    As for your ramblings about Gage R&R and transactional processes – you are misinformed.
    As for your last statement – I believe it betrays your true spirit.
    – Andy

    0
    #139212

    HF Chris
    Participant

    It is that old human factors mental error again. Usability and applicability. I hate when they keep showing up.
    HF Chris
     

    0
    #139214

    Heebeegeebee BB
    Participant

    Andy,
    It IS infact, the Customer who defines specifications and what constitutes a defect.
    It doesn’t matter if your process is pouring a cup of coffee, writing code, or building a thermonuclear warhead.   same-same.
    VoC drives all activity.   Until you recognize this, nothing productive can come from debating the minutae and floating examples…
    Seriously, The Customer is King.   If these 3 criteria are not ALL met, whether dealing with widgets, or with services/transactions, it is Non Value-added:
    1) Does the Customer care? (or are they willing to pay for it?)
    2) Was there a physical transformation to the product or service (or code)
    3) Was it done right the first time?
    All or none.   I don’t know if a Customer really wants to pay for glitchy software outside of a deliberate testing environment, or white-hat hack.

    0
    #139217

    Anonymous
    Guest

    Heebeegeebee,
    If you would like me to provide an example of where customers got it wrong I’ll be glad to oblige.
    In fact, as you well know there are several types of cutomers – both internal and external. Sometimes external customers don’t know enough about technology to anticipate what might happen in the market place. This happened to the world’s foremost manufacturer of image setter equipment a few years ago. It cost them about £10M.
    From this perspective – designers are also customers and my position is they ought to know when a system has a defect or not. Obviously, if a customer perceives a defect or problem it shold be addressed immediately – even if he/she uses the mobile phone to remove staples from documents.
    When a system has a defect it seems self evidence that the system defective – unless we’re changing that relationship as well. Therefore, as Lean/ Six Sigma professionals we ought to consider how well our theories match experience, rather than keeping what fits and throwing away experience that don’t.
    I believe recent discussions on this forum and Darth’s admonition about self-reflection are indeed necessary to improve the improvement process, which is why I posed this strawman for discussion.
    Of course, the alternative approach is to believe Six Sigma is perfect and all we have to do is regard it as some ‘Theory of Form’ and ignore all empirical evidence to the contrary.
    Thank you for your contribution.
    Cheers,
    Andy

    0
    #139223

    Szentannai
    Member

    Hi Andy,
    I would argue that the example you gave about querying O’Toole in a database is simply a defect. The customer expectation (whether stated in the requirements or not) will be to be able query for “any valid” name. If the requirements capture this correctly by enumerating all the characters that are possible in a “valid” name then this will be a defect where the SW did not implement the requirement correctly. If the requirements failed to capture this, then it will be a defect in the requirements.
    In either case what will happen is that the defect will be fixed and a patch or a new version of the SW will be delivered (eventually :) ).
    Personally I use the defect and bug terms interchangeably, but I could also agree with the interpretation that a defect is something where the system needs to exchanched (as in your phone example) and a bug is something where the SW will be exchanged (upgraded, patched).
    Regards
    Sandor

    0
    #139229

    Anil Saini
    Participant

    Hi,
    I would say that a bug is an error (undesirable deviation from customer’s requirements) in the software before it goes into production & defect is an error in the software after it goes into production.
    Software Development process ends at the time when the application is finally released for production. Any deviation from the requirements before this are taken care in the same process (developers do unit testing, SQA do the testing — find bugs and fix them). Just before the final release, the software is given to the customer for UAT (user acceptance testing) – the customer tests the beta – finds “defects” and sends his report back to the SD team. These defects are logged in as bugs and are fixed.
    Now, the software is released and if defects are reported by users, they are logged in as bugs by the SD team and fixed in the next team.
    What, I am trying to say here is that, the operational definition of a defect is “BUG” – the customer calls it “DEFECT” (VOC).
    Thanks

    0
    #139230

    Szentannai
    Member

    Hi,
    just two quick observations:
    I would be wary of the term “deviation from requirements” as a synonim for bugs or defects . In fact quite a number of bugs come from defects in the requirements – those guys writing them are only humans after all :-)).
    The other point I’d like to make is that SW development does NOT end at delivery. In a good quality product 60 to 80% of the development budget will be spent after the “final” delivery  for new versions, patches, bug-fixes etc.
    For a crappy product it will be much less, because no one will care to propose changes or report bugs.
    Regards
    Sandor

    0
    #139231

    Anil Saini
    Participant

    Sandor,
    I kind of agree with you on the first point. Regarding the second point – that’s were the catch is, I am considering each release/version/patch/hotfix/upgrade as a SD process (requirement to delivery). This enables you to keep a tab on the progress of the software towards defect/bug free state.
    Regards
    Anil

    0
    #139232

    Szentannai
    Member

    Hi Anil,
    I guess both ways of looking at it can be valid, depending on the circumstances. In a big project there are so many dependencies and limitations that depend on past history that I’d be reluctant to consider it a “new” process every time time we do a new release. I’ve also seen a small company go bankrupt because they thought that the development work ends at the final delivery – and did not budget for the 60-80% that came afterwards.
    On the other hand, for many purposes, like process improvement for example, it could be very helpful to see the development process the way you propose.
    regards
    Sandor

    0
    #139233

    Anonymous
    Guest

    Sandor/ Anil,
    I’d like to thank you both for entering into the spirit of this debate.
    Good luck!
    Andy

    0
Viewing 19 posts - 1 through 19 (of 19 total)

The forum ‘General’ is closed to new topics and replies.