An information technology (IT) services company was engaged by a European telecom service provider to provide end-to-end testing for its five lines of business. End-to-end testing is a methodology used to test whether the flow of a software application is functioning as expected from start to finish – for example, from receipt of customer order all the way through to fulfillment, accounting and shipping. This type of testing helps identify system dependencies and ensure that the right information is passed between various system components.
The end-to-end (E2E) test team soon discovered that each line of business within the telecom company had its own way of operating, including how it managed delivery, handled fluctuations in workload, reported on performance, and so on. The complexity of the situation was compounded by the number of internal and external teams the business lines interacted with, such as the design community, compotent teams and the project management office (PMO). The table below summarizes some of the challenges faced by the E2E team as they began their engagement with the client.
|Key Challenges in the Five Business Lines|
|Different delivery structure||
|Different processes for different lines of business||No standard operating procedures followed by the business lines.|
|Resource utilization||Demand fluctuations and lack of cross-skilled resources led to resource underutilization.|
|Handoffs between teams||Multiple handoffs between multiple teams led to complicated and time consuming process.|
|Different structures for business reports||Different sets of reports produced by each of the lines of business led to rework.|
To best manage the challenges noted above, the E2E test team decided to create in collaboration with the client company a “global test factory” – a dedicated space in which all E2E testing for the five business lines would take place. The factory would be built on Lean principles and shared services to enhance the client’s capabilities and thus exceed their performance expectations. (See Figure 1.) Its mission: to lead the way to the future of IT testing services by creating an adaptive organization through intelligent design.
The factory’s process owners (senior E2E team managers) identified the following key enablers needed to for the factory to fulfill its mission:
The company chose to apply Lean in order to reduce waste in the processes related to E2E testing. The elimination of waste from the processes would help reduce cycle times, reduce the cost of operations and improve performance. The IT company had experience with Lean, having already implemented the practice in the operational areas of fulfillment, assurance and billing.
The concerns noted in Figure 1 provided additional motivation to apply Lean principles to the global test factory to enhance E2E testing capabilities. The factory contained five “layers” (see Figure 2):
The factory team prepared a high-level project plan (Figure 3) and project charter (Figure 4). Deliverables were mapped after each milestone. The company approached the application of Lean to E2E testing by performing due diligence and identifying the non-value-added steps from the processes. This approach (as shown in Figure 3) was followed for each of the five layers of the test factory.
Figure 5 shows an example of one of the release management processes used in the factory. Release management is the process of managing new versions of the software, from the requirements-gathering stage to when software goes live. Releases are likely to include sprints (agreed-upon phases to design, develop and test) so that the particular software reaches the final stage in a controlled manner.
In the current-state VSM, several problem areas were identified:
Lean was used to develop the future-state value stream map (VSM). In the course of that exercise, the Lean implementation team eliminated 10 non-value-added steps from the release management process and made the following recommendations:
Similar to its work on the release management process, the factory’s Lean team also completed due diligence with other areas, such as demand management, front door (team that looks at the low-level details of the customer’s demands; a team within the E2E team), and test execution. Figure 6 shows some of the qualitative recommendations of the factory team which clearly tied recommendations for actions to solution owners.
The following qualitative benefits were achieved at the global test factory:
Quantitative benefits were calculated for all functions after NVA activities were removed, as shown in Figure 7. In total, the changes led to a savings of more than 4,400 hours over a nine-month period. Those 4,400 hours equated to 25 full-time employees; each person was reassigned to a different position within the company – no jobs were lost.