Since the start of the information technology age, software quality has been an ambiguous term, meaning different things to different people. It has been defined internally from the viewpoint of software developers; and it has been defined externally from the viewpoint of end users of the software. But either way, early in the history of software development it was obvious that software could not be developed in an ad hoc manner but needed to be subject to rules/methodologies to deliver a product or service that had attributes like robustness, correctness, adaptability, reusability and maintainability.
During the last couple of years numerous software development methodologies have been introduced to guide development teams in achieving these quality goals. Some methodologies involve a disciplined and detailed process with strong emphasis on planning. Examples of such methods include capability maturity model (CMM), software process improvement and capability determination model (SPICE), team software process (TSP), personal software process (PSP), etc. Then recently some Agile methods have been advocated as a new paradigm for high-speed, quick-to-market software development. Examples include extreme programming (XP), SCRUM, etc. The basic critique of the agile software development evangelists is that conventional software development processes are too rigid to achieve the end-results let alone the aimed quality factors for contemporary projects.
But one thing is sure: no matter which process/methodology is applied, the bottom line is that there is an impact of software processes on the software quality factors.
In what most consider a mature software/IT industry, the success/failure rate is not as good a one might expect. In fact, surveys by professional bodies like the Standish Group, Robbins-Gioia, KPMG, OASIG, etc. reveal a dismal picture. Some results from the 1995 Chaos report by the Standish Group found out the following:
Even though the numbers tended to improve with the later reports, the top reasons found for project failures are still the same:
All this leads to the conclusion that the business value of IT is still only vaguely perceived. And with the current watch on technology investments, there is even greater need for obtaining a good return on investment (ROI) on money spent on technology. A disciplined approach to software generation is quintessential if this goal is to be accomplished.
When one looks at an industry structure, it becomes apparent that the focus of the players depends on the relative competitive positioning. In a monopoly or an oligopoly (few key players), the companies typically look at their cost and add a mark-up (to get the sought-after market volume) to get to a certain selling price. The equation looks like:
Cost + Profit = Selling Price
But as the industry structure moves towards greater competition with many players providing almost a commodity product/service, the perspective of the equation changes:
Selling Price – Cost = Profit
It is a different paradigm when the players have similarly priced products. The only way to gain competitive advantage is to focus on minimizing the costs that reduce the bottom line. Minimizing costs is exactly where the IT and application development industry is. The current critical nature of software and application development in a product or service offering has elevated the importance of the software process in ROI considerations. Especially for any new IT initiative, there is a lot of effort to insure a satisfactory return on investment.
It can be a challenge to summarize the cost and benefit factors and values that contribute to estimating ROI in any project. This is where Six Sigma offers a set of tools for tracking and enabling success. The Six Sigma value proposition is directly tied to positive financial results, and hence the methodology often has the buy-in of top management. This focus on financial results ensures that executive buy-in remains through the execution and rollout phase of a project, the lack of which is a major reason of failure of IT projects. It also ensures business discipline in project selection and project tracking, with a focus on maximizing the benefits delivered to the bottom line. The Six Sigma philosophy is that satisfied customers come back for more and encourage their business associates, family and friends to do the same. It recognizes that the ultimate quality award is improved bottom-line profitability.
As suggested in Juran’s Quality Handbook, analysis of many application development projects has consistently revealed two facts:
Hence, cost savings can be realized only with methodologies that yield high quality initially and manage the change control process. This becomes even more relevant with the recent emphasis on a fundamental decision-support tool called “total cost of ownership.” This tool requires a holistic view of all costs across the organization that will be incurred throughout the life cycle of the project, including people, processes and technologies.
The typical cost drivers of a Six Sigma initiative for software process improvement that are essential inputs into the ROI estimating process, include:
Typical benefits of such an effort include:
Six Sigma practitioners follow these tenets as a business philosophy:
And so Six Sigma, in general, measures deviations from an ideal. The measurable thing may be the process used to make a toothbrush, whose material specifications and costs are measured and compared to ideal material specifications and ideal cost. Or it may be the process of a stock brokerage, where quality of service and cost of service are measured against ideals of quality and service. When it is applied to IT, Six Sigma measures attributes like network speed, network reliability and relevant line of business processes.
Thus, measurements are made to gain an understanding of processes to see what works and what does not; to evaluate, predict and determine status with respect to plans; and to improve (rational use of quantitative information to identify problems and strategies to remove them).
Six Sigma started in the manufacturing industry with emphasis on management of efficient processes, efficient management of people, dedication to measurement systems, etc. But it became apparent that business success was more than the absence of negatives (defects, delays, cost overruns). Six Sigma then began to encompass positives like customer loyalty and delighters in new products. From operational excellence, Six Sigma has moved towards customer intimacy and product leadership value disciplines.
There are essentially two approaches to ensure software quality:
There is always need to study both the quality of the software product and the life-cycle cost incurred in the development and maintenance of the products. The quality of software has been studied mainly from defect analysis and software maintenance perspectives. The effect of the process used in a software project on the outcome of the project in terms of cost to the software developer and quality of the product is key. And this is where Six Sigma for software comes in. It hold that the process of generating the product is more important than the product itself.
But an IT person will immediately point out, application development is not like manufacturing since process variation can never be eliminated because:
Yet the software application development processes can be fully characterized by three simple measurements:
Six Sigma for software is not a software development process definition – instead it is a general methodology for improving processes and products. It provides a holistic quality objective in product development, combining a programmer’s desire for bug-free piece of software and a user’s desire for an easy-to-use application that fulfills a business need. Although a few elements of the Six Sigma toolkit are invoked within other methodologies like the Software Engineering Institute’s PSP/TSP framework (e.g., regression analysis for development of estimating models), there are many other tools available in the Six Sigma for generating good quality software products.
For instance, voice of the customer (VOC) and quality function deployment (QFD) are useful for developing detailed and prioritized customer requirements (both stated and unstated). More than just sending out survey forms, these tools involve creating meaningful customer feedback. VOC is captured in a variety of ways – direct interviews, surveys, focus groups, customer specifications, observations, etc. This understanding of the customer needs is then summarized in a matrix or QFD. These tools ensure that the following necessary steps are taken care of:
Then there are numerous charting/calculation techniques that can be used to scrutinize cost, schedule and quality (project-level and personal-level) data as a project proceeds. For technical development, there are quantitative methods for risk analysis and concept/design selection.
Once management has driven the business strategy down to projects, one can work on them with the Six Sigma DMAIC approach – define, measure, analyze, improve, control. This starts with the assumption that all work is a process. And the goal is to improve the process in general. But Six Sigma applications in software also reach beyond the business goal of improving current processes/products and extend to design of new processes/products. Six Sigma incorporates the Design for Six Sigma (DFSS) approach to improving the feature/function/cost trade-off in definition and design of the software product. Tools such as KJ analysis, QFD, conjoint analysis, design of experiments (DOE) and many others have high leverage applications in the world of new product development in software.
There are many models that have been used for disciplined application development and the key to understand is that the disparate quality models need to be evaluated in the context of business objectives. Project characteristics that determine the relevance of a particular methodology can be broadly based on following dimensions:
Project Size: This is not only the measure of lines of code or megabytes of data, but primarily the complexity of the system, the nature of its main components, interactions and – importantly – the size and composition of the development team.
Application Domain: This includes not only the domain of environmental management but also the characteristics in more technical terms – real-time, distributed (client/server architecture with both LAN and WAN elements), decision support objective, etc.
Criticality: Projects of high criticality like the ones doing forecasting for weather conditions, or decision-support systems in important domains like emergency healthcare need rigorous methodologies to ensure the quality.
Innovation: If a project is a research project, involving a number of new and partly not yet proven technologies, like 3D dynamic simulation, a fair amount of process customization can be done to eliminate need to follow rigid procedures.
Figure 2 provides a matrix that suggests a way of selecting a model for a certain project.
Heuristically, it has been found in the application development industry that Six Sigma for software begins to be cost-effective once the organization has reached CMM Levels of 4 or 5. As an organization matures to those levels, it begins to truly leverage established measurement practices, and that is when accomplishment of true Six Sigma performance becomes a relevant goal. And at that point, Six Sigma and approaches such as CMM/CMMI, PSP and TSP become complementary and mutually supportive. From a corporate strategy viewpoint, Six Sigma is an enabler to launch CMM, CMMI, PSP, etc. with similar tools, methods and concepts of continuous improvement. And yet it differs from CMM/CMMI in that:
The most fundamental tenet of Six Sigma is that the business must be “managed manage by fact.” This view is consistent with that of TSP/PSP, but it has not yet been established that PSP/TSP is the best alternative in every context, only that it is better than some alternatives. Six Sigma can help organizations find the solutions that are truly optimal for each unique circumstance.
As information technology becomes more involved as a business value driver in different industries, there will be improvements in IT’s own business processes. This will necessitate IT to become part of strategic processes and use Six Sigma concepts like business initiative evaluation, prioritization and approval, IT resource supply-and-demand balancing process, etc.