The Benefits of Metrics
A metric is a standard unit used to collect and report performance of a particular IT service. While metrics can require considerable time and effort to develop and then to report regularly, there are many reasons to make the investment. For IT leaders, being able to recognize and develop effective metrics is becoming an increasingly important competency.
IT metrics help IT leaders by:
- Presenting hard evidence to business executives to justify requests for funding and resource levels for a project and to gain top-level management buy-in.
- Highlighting the rationale behind any increase in IT spending during the course of a project.
- Providing IT project managers with tools to track progress of projects and identify potential pitfalls.
- Helping with project prioritization.
- Demonstrating the business value of IT, which may include cost efficiency and process efficiency, to senior management.
- Defining directions for continuous improvement in response to customers’ demands.
The Harsh Reality
A recent Info-Tech survey on “Metrics and Benchmarking” with nearly 1,000 responses found that:
- Close to half of respondents indicated they are very interested in metrics.
- However, when it comes to spending the time and effort to collect the data needed to provide comparative information (aka benchmarks) only 12% of respondents were willing to commit to doing so.
Figure 1. Clients’ Interest in Metrics
Source: Info-Tech Research Group
Figure 2. Clients’ Commitment to Provide Data for Metrics
Source: Info-Tech Research Group
Recommendations
1.Measure the right things. For IT leaders, IT metrics serve two objectives:
-
To monitor and measure performance of technology, IT processes and projects, and
-
To communicate the performance and value of IT services to business stakeholders.
It is important when deciding what to measure to clearly consider what the IT organization uses to measure its own performance and what metrics are understandable by the business. For example, IT may track a technology performance metric such as Mean-Time-Before-Failure (MTBF) for storage arrays. This is a metric that IT uses to monitor the performance of technology and helps to inform future vendor evaluation.
However, the business community is unlikely to relate to MTBF as a measure of IT service quality. Rather a metric such as Application Availability (usually expressed as the percentage of time an application is available to appropriate users during normal business hours) will be more relevant to business activities. Be sure to focus on the services that the business uses and select metrics that make sense to IT users. For more information, refer to the McLean Report research note, “Seven Key Metrics for Professional Services.”
2. Design quality metrics. The most effective metrics are SMART: An example of a SMART metric is Precentage of IT expenses for handheld devices in hospitals.
-
Specific and targeted to the area of measurement- Only targeting mobile devices such as PDAs, pagers, blackberrys used by doctors and nurses to retrieve patients’ files.
-
Measurable, so that accurate and complete data can be collected- Data can be easily collected on the purchase and maintenance cost of each device compared to the total IT expense.
-
Actionable, in that metrics are clear in direction and easy to understand- IT leaders can use this metric to take action when the percentage IT expense for handheld devices exceeds the expected limit.
-
Relevant, in that they only measure the things which are important-As the health care industry is moving towards electronic health record systems, handheld devices represent a significant percentage of IT budget; therefore, this will be a very useful metric to report on.
-
Timely metrics are those in which the data is available when required- Tracking the trending of this metric on a monthly basis might show gradual increase in the expenditure and thus provides timely information before it exceeds the threshold of IT budget allocated particularly for this purpose.
3. Design IT metrics that deliver business value. Traditionally, IT has been considered a cost center. However, IT is continuously trying to improve business value by acting as a service provider and IT leaders are using different metrics to back this up. Yet, business management remains skeptical about the growth in IT spending and likes to understand how IT’s objectives are aligned with the organization’s objectives before they approve any budget increase.
Metrics designed to accurately measure and account for the impact of technology investments on business value are very important. When preparing an IT scorecard, define the business objective for each metric. Involve business units to find out what they consider useful performance indicators. For more information on how to design an IT performance scorecard refer to the McLean Report research note, “Use the Right Metrics When Building an IT Performance Scorecard.”
4. Collect relevant data. Two risks arise when the time comes to gather the data needed to determine metrics. With the availability of advanced systems management tools, there is a tremendous amount of data for hardware performance, network availability, software performance and many other components. The mere fact the data exists may cause IT organizations to analyze it and create metrics that overwhelm the user community. Furthermore, these numbers fail to identify the root cause of an issue the IT department is working to resolve.
A second risk is the lack of automated tools to repeatedly collect data and calculate metrics on a regular basis. In this case, data collection is a manually intensive process and the effort to collect data points quickly begins to outweigh the perceived benefits of the metrics themselves.
In both situations, the key is to focus on the data which are absolutely necessary and relevant to manage IT performance and to communicate to business stakeholders.
5. Commit to provide data to obtain high quality metrics. Using third-party resources, such as Info-Tech, to compare metrics to peer organizations is common practice. However, the quality of this source of information is limited by the quality of data provided by survey respondents. Select the source of benchmark data carefully and evaluate the quality of the underlying data and its relevance to key metrics. When requested by the third party to participate in a metrics survey, strongly consider committing the time necessary to provide quality data at the right level of detail.
Bottom Line
When the pressure is on to provide predictable, reliable and valuable support to business operations, IT leaders can back up performance claims with hard data in the form of metrics. Be aware of the characteristics of good metrics, how to capture them and how to best report them. Follow Info-Tech’s five recommendations to become a metrics master.
