Several information technology (IT) metrics can be developed on the basis of best practice frameworks such as capability maturity model integration (CMMI) and information technology infrastructure library (ITIL). Measuring and collecting such data, which is often part of process improvement initiatives, brings to light what is happening in IT processes.
Following are a few tips for metrics definition and deployment. These tips are divided into the strategic, tactical and operational aspects of metrics.
One reason a metrics program is beneficial is because it helps IT collect voice-of-the-customer data. For instance, an IT practitioner might ask internal customers: What are the areas within IT where the business leadership is trying to gain insight? What are the motives to focus on those areas?
Practitioners do not want to risk following the wrong metrics. Designing a dashboard with a wrong set of metrics may mean a scrapped project, as leadership may soon realize what they obtained from the metrics initiative is not what they wanted in the first place.
A detailed study, starting with interviews with the leadership, is necessary. The practitioner must articulate the needs behind the metrics initiative, and verify them with leadership, before defining the individual metrics.
It is also important that practitioners do not force mature metrics, such as function points, starting on Day 1. Asking for high-maturity metrics will not help if the organization does not have the proper measurement framework and the resources to collect it. Until and unless the leadership and the project teams understand the needs and benefits of a particular measure, and are assured that the effort needed to gather them will be beneficial, resistance may remain high.
Suppose the leadership wants to see metrics around productivity, but the organization does not count function points. If the practitioner tries to enforce function point count or lines of code count within project teams, deployment may face the risk of failure because of too many protesters from project teams. Instead, practitioners can create the need for the high maturity metric by depicting how that metric would fill a missing niche. It may help to start with simpler metrics, such as lines of code, and demonstrate how they yield crude productivity measures. However, every report should carry the message that productivity calculation may be improved greatly with the help of function points.
One key aspect of metrics collection is the choice of unit. In order to collect project metrics, practitioners may ask the process owners to provide defect counts in a data collection template that is assigned to the project. Although a less time-consuming way of collecting defect information, the approach has two caveats:
On the other hand, if a practitioner asks the process owners to provide the log entry for each defect – including resolution time, effort spent, brief description and unique identification number – different metrics can be generated from the information and the entry would be verifiable from the system.
Activities in IT projects or programs, and the metrics related to them, may broadly be divided into three categories: development, maintenance/enhancement and production support. Accordingly, the unit of work varies, and so does the priority of metrics. But the overall theme of metrics, in absence of any special considerations, covers:
For a metrics rollout across an organization, the deployment needs to be scheduled into small steps instead of planning for one large rollout on a certain day. Communications to all employees, such as “Start collecting metrics from Nov. 1, 2008,” might arouse doubts and resistance. The deployment needs to be staged across the groups with a span of one to three months, and deployed team by team.
Following are five steps to carry out for each team in a metrics deployment:
In order to bring more enthusiastic project leaders under the metrics program voluntarily, remember to share success stories in newsletters and mailers from leadership teams.
While conducting an introductory session with a process owner, tell them how another process owner expressed enthusiasm after seeing the metrics for their team and show the results too, if possible. Process owners may look at metrics as overhead initially, but providing them the monthly metrics with interpretations will turn them to enthusiasts, rather than resistors.