Companies today make great financial investments in testing their processes and systems to ensure that they will meet customer expectations. The companies go through all the right steps to test as thoroughly as possible; but – in many cases – the end results are less than desired for the effort, time and dollars expended. Customer experiences do not meet expectations.

There is a high degree of variability in the quality and effectiveness of testing. This variability is caused by:

1. Lack of Resources

• Not having people who are dedicated and knowledgeable about processes, applications and testing procedures
• Not having hardware and software configurations to support the testing processes
• Not having tools such as deliverable templates, change control, version management and automated testing

2. Flawed Processes

• The testing process does not test what was developed to fulfill the customer request
• Existing testing processes do not account for defined gating, hand-off, sign-off, types of testing and planning
• Project portfolio management does not account for resource prioritization, initiatives dependencies and overlaps
• The process is re-created each time a test is conducted, test scripts are not archived for future use, and validation criteria are lost

3. Channel Implications

• New and ancillary projects exhibit the highest degree of variability
• Multi-channel initiatives exhibit the greatest complexity and challenges
• Mainframe-centric projects exhibit the greatest maturity

Implementation of the following best practices starts to address the variability issues:

• On-going process improvement capability
• Dedicated process performers, tools and environments to manage the testing process
• Coordination within the project portfolio and other initiatives

Given the causes of variability and the need for continuous improvement, it only makes sense to introduce in the development cycle a value proposition that brings the testing process under the DMAIC (Define, Measure, Analyze, Improve, Control) method.

Value Proposition: Project Portfolio Management
Define	Measure	Analyze	Improve		Control
Knowledge of business problem/request	Testing magnitude/requirements	Test cases and plan	Systems integration testing	Testing performed resulting in a business-as-usual environment	Certified business-as-usual environment with metrics

This value proposition maps the traditional testing process to DMAIC and supports successful development and implementation of new products and services for customers. The DMAIC approach also ensures completeness of testing. During the Define phase, knowledge is gained about the business problem/request. In the Measure phase, the testing magnitude/requirements are determined. The test cases and plan are developed during the Analyze phase. In the Improve phase, the testing is completed to provide a business-as-usual environment.

The systems integration testing (SIT) work is recognized as being an effort that is owned and managed by the Technology Group during the Improve phase. The Technology Group in most instances is using ISO 9000 or the capability maturity model to conduct its project execution. The Control phase is where continuous improvement is done based on data that is captured and tracked to assure the business-as-usual environment is operating within the desired control parameters.

The Project Portfolio Management is the umbrella where all change initiatives are prioritized. If a priority is changed, the business decision is known and the impact on the customer request is taken into account. The Portfolio Management Group understands the channel variability. It knows whether the request is going to be new and ancillary, multi-channel or mainframe-based.

Defined testing roles are used to execute the testing process. Each phase of the DMAIC process is initiated by defined inputs and outputs. Each phase goes through a tollgate process to determine if the next phase should be initiated. All of the actions are aimed at eliminating the variations outlined above.

The success rate in implementing the business-as-usual environment is high. This success has come from knowing the metrics that are going to be used to measure the variations of the new process once it is into day-to-day production. Then using these metrics for measurement during the improve phase.

Inputs and Outputs in DMAIC
	Testing Role
	Define	Measure	Analyze	Improve	Control
Sales	TBD	TBD	TBD	TBD	TBD
Change Management (Testing)	Responsible/ accountable	Responsible/ accountable	Responsible/ accountable	Responsible	Responsible/ participate
Customer	Participate	Participate	Participate	Accountable	Approve
Quality	Participate	Participate	Participate	Participate	Participate
Technology	Participate	Participate	Participate	Support	Support
Learning	TBD	TBD	TBD	TBD	TBD
Finance	TBD	TBD	TBD	TBD	TBD
Activities	· Understand business requirement · Define client participation · Define metrics	· Gather and document testing requirements · Create calendar · Gather costs · Determine resource needs	· Analyze dependencies, impacts and needs · Define depth, dimensions of testing · Update calendar · Transform BRD into test cases and sync HLD · Create test cases and data · Create and finalize test plan	· Update calendar · Allocate resources for testing · Modify test plan, test data and cases · Testing change control-calendar, cycles, versions, and incident management · Perform tests · Dress rehearsal · Metric reporting	· Certify operating environment · Metric reporting
Outputs	· Knowledge of business request · Metrics	· Testing requirements · Calendar · Budget · Resources	· Test plan, testing depth and dimension · Impacts · Test cases · Test data · Executable test plan	· Test plan, data, cases, calendar and resources · Metrics reports · Business-as-usual platforms	Metrics
Tollgate reviews occur between each phase of the DMAIC process

Defining Roles and Action Items

Testing business processes must include the lines of business that will actually execute those processes. It is critical that roles and responsibilities be assigned to ensure the testing role validates that process and technology meets a company’s CTQ requirements. Here is a quick primer for the action definitions used in the Testing Role chart:

Responsible: Ensures the activity is completed
Accountable: Integrates the deliverable/activity to provide a complete product for the customer
Participate: Prepares, plans, designs and builds, delivers specific outputs
Knowledge: Aware of activities to communicate with the customer
Support: Provides day-to-day, end-to-end operational production environment for customer
Approve: Provides sign-off

Experience shows that numerous areas within a company may be involved in the testing process based on the type of change. Sometimes an area will be involved and other times it will not. The chart above recognizes the areas and also identifies the actions that the areas will be taking.

The Change Management area is the driver of the testing process. It provides the tools to execute the process: deliverable templates, change control, version management and automated testing.

The company ensures that the proper test cases are developed, conducted and validated.

The Technology area addresses the hardware and the software configurations to assure that the platforms used for the testing will be near similar to the ones that will support the production processes.

The DMAIC approach allows the company to address the variability of the testing process with a prescription for success. The phases allow for: 1) resource identification, knowledge and availability, 2) processes to test to assure that the customer request is fulfilled and 3) channel implications: something well-known (low risk) or something new (higher risk). The overall measurement has allowed delivery of business-as-usual environments that meet the metrics as defined by the customer and place the metrics in a day-to-day environment for ongoing measurement and improvement.