Get Instant Access
to This Blueprint

Cio icon

Develop Meaningful Service Metrics

Reinforce service orientation in your IT organization by ensuring your IT metrics generate value-driven resource behavior.

  • IT organizations measure services from a technology perspective but rarely from a business goal or outcome perspective.
  • Most organizations do a poor job of identifying and measuring service outcomes over the duration of a service’s lifecycle – never ensuring the services remain valuable and meet expected long-term ROI.

Our Advice

Critical Insight

  • Service metrics are critical to ensuring alignment of IT service performance and business service value achievement.
  • Service metrics reinforce positive business and end-user relationships by providing user-centric information that drives responsiveness and consistent service improvement.
  • Poorly designed metrics drive unintended and unproductive behaviors that have negative impacts on IT and produce negative service outcomes.

Impact and Result

Effective service metrics will provide the following service gains:

  • Confirm service performance and identify gaps.
  • Drive service improvement to maximize service value.
  • Validate performance improvements while quantifying and demonstrating business value.
  • Ensure service reporting aligns with end-user experience.
  • Achieve and confirm process and regulatory compliance.

Which will translate into the following relationship gains:

  • Embed IT into business value achievement.
  • Improve the relationship between the business and IT.
  • Achieve higher customer satisfaction (happier end users receiving expected service, the business is able to identify how things are really performing).
  • Reinforce desirable actions and behaviors from both IT and the business.

Develop Meaningful Service Metrics Research & Tools

Start here – read the Executive Brief

Read our concise Executive Brief to find out why you should develop meaningful service metrics, review Info-Tech’s methodology, and understand the four ways we can support you in completing this project.

1. Design the metrics

Identify the appropriate service metrics based on stakeholder needs.

2. Design reports and dashboards

Present the right metrics in the most interesting and stakeholder-centric way possible.

3. Implement, track, and maintain

Run a pilot with a smaller sample of defined service metrics, then continuously validate your approach and make refinements to the processes.


Member Testimonials

After each Info-Tech experience, we ask our members to quantify the real-time savings, monetary impact, and project improvements our research helped them achieve. See our top member experiences for this blueprint and what our clients have to say.

10.0/10


Overall Impact

$1,673


Average $ Saved

57


Average Days Saved

Client

Experience

Impact

$ Saved

Days Saved

Government of Bermuda

Guided Implementation

10/10

N/A

N/A

Altarum Institute

Guided Implementation

10/10

N/A

1

Shelby County Government

Workshop

10/10

N/A

120

Dollar General

Guided Implementation

10/10

$2,479

50

Open Text Corporation

Guided Implementation

9/10

N/A

5

University of Ottawa

Workshop

8/10

$25,000

14

Tulane University

Guided Implementation

10/10

$14,259

50

Office of the Attorney General

Guided Implementation

10/10

$12,292

10

Allens Operations Pty Ltd.

Guided Implementation

8/10

N/A

N/A

Akin Gump Strauss Hauer & Feld LLP

Guided Implementation

9/10

N/A

6

University of Wisconsin-Madison

Guided Implementation

10/10

$99,166

60

Clark Pacific

Guided Implementation

9/10

$509K

N/A


Performance Measurement

Measure IT right and measure it well.
This course makes up part of the Strategy & Governance Certificate.

Now Playing: Academy: Performance Measurement | Executive Brief

An active membership is required to access Info-Tech Academy
  • Course Modules: 4
  • Estimated Completion Time: 1.5-2 hours
  • Featured Analysts:
  • Gord Harrison, Senior Vice President, Research
  • Valence Howden, Principal Research Director, CIO Practice

Workshop: Develop Meaningful Service Metrics

Workshops offer an easy way to accelerate your project. If you are unable to do the project yourself, and a Guided Implementation isn't enough, we offer low-cost delivery of our project workshops. We take you through every phase of your project and ensure that you have a roadmap in place to complete your project successfully.

Module 1: Design the Metrics

The Purpose

  • Define stakeholder needs for IT based on their success criteria and identify IT services that are tied to the delivery of business outcomes.
  • Derive meaningful service metrics based on identified IT services and validate that metrics can be collected and measured.

Key Benefits Achieved

  • Design meaningful service metrics from stakeholder needs.
  • Validate that metrics can be collected and measured.

Activities

Outputs

1.1

Determine stakeholder needs, goals, and pain points.

  • Understand stakeholder priorities
1.2

Determine the success criteria and related IT services.

  • Adopt a business-centric perspective to align IT and business views
1.3

Derive the service metrics.

  • Derive meaningful business metrics that are relevant to the stakeholders
1.4

Validate the data collection process.

  • Determine if and how the identified metrics can be collected and measured
1.5

Validate metrics with stakeholders.

  • Establish a feedback mechanism to have business stakeholders validate the meaningfulness of the metrics

Module 2: Design Reports and Dashboards

The Purpose

  • Determine the most appropriate presentation format based on stakeholder needs.

Key Benefits Achieved

  • Ensure the metrics are presented in the most interesting and stakeholder-centric way possible to guarantee that they are read and used.

Activities

Outputs

2.1

Understand the different presentation options.

  • Learn about infographic, scorecard, formal report, and dashboard presentation options
2.2

Assess stakeholder needs for information.

  • Determine how stakeholders would like to view information and how the metrics can be presented to aid decision making
2.3

Select and design the metric report.

  • Select the most appropriate presentation format and create a rough draft of how the report should look

Module 3: Implement, Track, and Maintain Your Metrics

The Purpose

  • Run a pilot with a smaller sample of defined service metrics to validate your approach.
  • Make refinements to the implementation and maintenance processes prior to activating all service metrics.

Key Benefits Achieved

  • High user acceptance and usability of the metrics.
  • Processes of identifying and presenting metrics are continuously validated and improved.

Activities

Outputs

3.1

Select the pilot metrics.

  • Select the metrics that should be first implemented based on urgency and impact
  • Complete the service intake form for a specific initiative
3.2

Gather data and set initial targets.

  • Create a process to gather data, measure baselines, and set initial targets
3.3

Generate the reports and validate with stakeholders.

  • Establish a process to receive feedback from the business stakeholders once the report is generated
3.4

Implement the service metrics program.

  • Identify the approach to implement the metrics program across the organization
3.5

Track and maintain the metrics program.

  • Set up mechanism to ensure the success of the metrics program by assessing process adherence and process validity

Develop Meaningful Service Metrics

Select IT service metrics that drive business value.

ANALYST PERSPECTIVE

Are you measuring and reporting what the business needs to know?

“Service metrics are one of the key tools at IT’s disposal in articulating and ensuring its value to the business, yet metrics are rarely designed and used for that purpose.

Creating IT service metrics directly from business and stakeholder outcomes and goals, written from the business perspective and using business language, is critical to ensuring that the services that IT provides are meeting business needs.

The ability to measure, manage, and improve IT service performance in relation to critical business success factors, with properly designed metrics, embeds IT in the value chain of the business and ensures IT’s focus on where and how it enables business outcomes.”

Valence Howden,
Senior Manager, CIO Advisory
Info-Tech Research Group

Our understanding of the problem

This Research Is Designed For:
  • CIO
  • IT VPs
This Research Will Help You:
  • Align business/IT objectives (design top-down or outside-in)
  • Significantly improve the relationship between the business and IT aspects of the organization
  • Reinforce desirable actions and behaviors
This Research Will Also Assist:
  • Service Level Managers
  • Service Owners
  • Program Owners
This Research Will Help Them
  • Identify unusual deviations from the normal operating state
  • Drive service improvement to maximize service value
  • Validate the value of performance improvements while quantifying and demonstrating benefits realization

Executive summary

Situation

  • IT organizations measure services from a technology perspective yet rarely measure services from a business goal/outcome perspective.
  • Most organizations do a poor job of identifying and measuring service outcomes over the duration of a service’s lifecycle – never ensuring the services remain valuable and meet expected long-term ROI.

Complication

  • IT organizations have difficulty identifying the right metrics to demonstrate the value of IT services to the business in tangible terms.
  • IT metrics, as currently designed, reinforce division between the IT and business perspectives of service performance. They drive siloed thinking and finger-pointing within the IT structure, and prevent IT resources from understanding how their work impacts business value.

Resolution

  • Our program enables IT to develop the right service metrics to tie IT service performance to business value and user experience.
  • Ensure the metrics you implement have immediate stakeholder value, reinforcing alignment between IT and the business while influencing behavior in the desired direction.
  • Make sure that your metrics are defined in relation to the business goals and drivers, ensuring they will provide actionable outcomes.

Info-Tech Insight

  1. Service metrics are critical to ensuring alignment of IT service performance and business service value achievement.
  2. Service metrics reinforce positive business and end-user relationships by providing user-centric information that drives responsiveness and consistent service improvement.
  3. Poorly designed metrics drive unintended and unproductive behaviors, which have negative impacts on IT and produce negative service outcomes.

Service metrics 101

What are service metrics?

Service metrics measure IT services in a way that relates to a business outcome. IT needs to measure performance from the business perspective using business language.

Why do we need service metrics?

To ensure the business cares about the metrics that IT produces, start with business needs to make sure you’re measuring the right things. This will give IT the opportunity talk to the right stakeholders and develop metrics that will meet their business needs.

Service metrics are designed with the business perspective in mind, so they are fully aligned with business objectives.

Perspectives Matter

Different stakeholders will require different types of metrics. A CEO may require metrics that provide a snapshot of the critical success of the company while a business manager is more concerned about the performance metrics of their department.

What are the benefits of implementing service metrics?

Service metrics help IT communicate with the business in business terms and enables IT to articulate how and where they provide business value. Business stakeholders can also easily understand how IT services contribute to their success.

The majority of CIOs feel metrics relating to business value and stakeholder satisfaction require significant improvement

A significantly higher proportion of CIOs than CEOs feel that there is significant improvement necessary for business value metrics and stakeholder satisfaction reporting. Stacked horizontal bar chart presenting survey results from CIOs and CXOs of 'Business Value Metrics'. Answer options are 'Effective', 'Some Improvement Necessary', 'Significant Improvement Necessary', and 'Not Required'.N=364

Stacked horizontal bar chart presenting survey results from CIOs and CXOs of 'Stakeholder Satisfaction Reporting'. Answer options are 'Effective', 'Some Improvement Necessary', 'Significant Improvement Necessary', and 'Not Required'.N=364

(Source: Info-Tech CIO-CXO Alignment Diagnostic Survey)

Meaningless metrics are a headache for the business

A major pitfall of many IT organizations is that they often provide pages of technical metrics that are meaningless to their business stakeholders.

  1. Too Many MetricsToo many metrics are provided and business leaders don’t know what to do with these metrics.
  2. Metrics Are Too TechnicalIT provides technical metrics that are hard to relate to business needs, and methods of calculating metrics are not clearly understood, articulated, and agreed on.
  3. Metrics Have No Business ValueService metrics are not mapped to business goals/objectives and they drive incorrect actions or spend.
When considering only CEOs who said that stakeholder satisfaction reporting needed significant improvement, the average satisfaction score goes down to 61.6%, which is a drop in satisfaction of 12%.

A bar that says 73% dropping to a bar that says 61%. Description above.

(Source: Info-Tech Research Group CIO-CXO Alignment Diagnostic Survey)

Poorly designed metrics hurt IT’s image within the organization

By providing metrics that do not articulate the value of IT services, IT reinforces its role as a utility provider and an outsider to strategic decisions.

When the CIOs believe business value metrics weren’t required, 50% of their CEOs said that significant improvements were necessary.

Pie Chart presenting the survey results from CEOs regarding 'Business Value Metrics'. Description above.

(Source: Info-Tech Research Group CIO-CXO Alignment Diagnostic Survey)
  1. Reinforce the wrong behaviorThe wrong metrics drive us-against-them, siloed thinking within IT, and meeting metric targets is prioritized over providing meaningful outcomes.
  2. Do not reflect user experienceMetrics don’t align with actual business/user experience, reinforcing a poor view of IT services.
  3. Effort ≠ ValueInvesting dedicated resources and effort to the achievement of the wrong metrics will only leave IT more constrained for other important initiatives.

Articulate meaningful service performance that supports the achievement of business outcomes

Service metrics measure the performance of IT services and how they enable or drive the activity outcomes.

A business process consists of multiple business activities. In many cases, these business activities require one or more supporting IT services.

A 'Business Process' broken down to its parts, multiple 'Business Activities' and their 'IT Services'. For each business process, business stakeholders and their goals and objectives should be identified.

For each business activity that supports the completion of a business process, define the success criteria that must be met in order to produce the desirable outcome.

Identify the IT services that are used by business stakeholders for each business activity. Measure the performance of these services from a business perspective to arrive at the appropriate service metrics.

Differentiate between different types of metrics

Stakeholders have different goals and objectives; therefore, it is critical to identify what type of metrics should be presented to each stakeholder.

Business Metrics

Determine Business Success

Business metrics are derived from a pure business perspective. These are the metrics that the business stakeholders will measure themselves on, and business success is determined using these metrics.

Arrow pointing right.

Service Metrics

Manage Service Value to the Business

Service metrics are used to measure IT service performance against business outcomes. These metrics, while relating to IT services, are presented in business terms and are tied to business goals.

Arrow pointing right.

IT Metrics

Enable Operational Excellence

IT metrics are internal to the IT organization and used to manage IT service delivery. These metrics are technical, IT-specific, and drive action for IT. They are not presented to the business, and are not written in business language.

Implementing service metrics is a key step in becoming a service provider and business partner

As a prerequisite, IT organizations must have already established a solid relationship with the business and have a clear understanding of its critical business-facing services.

At the very least, IT needs to have a service-oriented view and understand the specific needs and objectives associated with each stakeholder.

Visualization of 'Business Relationship Management' with an early point on the line representing 'Service Provider: Establish service-oriented culture and business-centric service delivery', and the end of the line being 'Strategic Partner'.

Once IT can present service metrics that the business cares about, it can continue on the service provider journey by managing the performance of services based on business needs, determine and influence service demand, and assess service value to maximize benefits to the business.

Which processes drive service metrics?

Both business relationship management (BRM) and service level management (SLM) provide inputs into and receive outputs from service metrics.

Venn Diagram of 'Business Relationship Management', 'Service Metrics', and 'Service Level Management'.

Business Relationship Management

BRM works to understand the goals and objectives of the business and inputs them into the design of the service metrics.

Service Metrics

BRM leverages service metrics to help IT organizations manage the relationship with the business.

BRM articulates and manages expectations and ensures IT services are meeting business requirements.

Which processes drive service metrics?

Both BRM and SLM provide inputs into and receive outputs from service metrics.

Venn Diagram of 'Business Relationship Management', 'Service Metrics', and 'Service Level Management'.

Service Level Management

SLM works with the business to understand service requirements, which are key inputs in designing the service metrics.

Service Metrics

SLM leverages service metrics in overseeing the day-to-day delivery of IT services. It ensures they are provided to meet expected service level targets and objectives.

Effective service metrics will deliver both service gains and relationship gains

Effective service metrics will provide the following service gains:

  • Confirm service performance and identify gaps
  • Drive service improvement to maximize service value
  • Validate performance improvements while quantifying and demonstrating business value
  • Ensure service reporting aligns with end-user experience
  • Achieve and confirm process and regulatory compliance
      Which will translate into the following relationship gains:
      • Embed IT into business value achievement
      • Improve relationship between the business and IT
      • Achieve higher customer satisfaction (happier end users receiving expected service, the business is able to identify how things are really performing)
      • Reinforce desirable actions and behaviors from both IT and the business

Don’t let conventional wisdom become your roadblock

Conventional Wisdom

Info-Tech Perspective

Metrics are measured from an application or technology perspective Metrics need to be derived from a service and business outcome perspective.
The business doesn’t care about metrics Metrics are not usually designed to speak in business terms about business outcomes. Linking metrics to business objectives creates metrics that the business cares about.
It is difficult to have a metrics discussion with the business It is not a metrics/number discussion, it is a discussion on goals and outcomes.
Metrics are only presented for the implementation of the service, not the ongoing outcome of the service IT needs to focus on service outcome and not project outcome.
Quality can’t be measured Quality must be measured in order to properly manage services.

Our three-phase approach to service metrics development

Let Info-Tech guide you through your service metrics journey

1

2

3

Design Your Metrics Develop and Validate Reporting Implement, Track, and Maintain
Sample of Phase 1 of Info-Tech's service metric development package, 'Design Your Metrics'. Sample of Phase 2 of Info-Tech's service metric development package, 'Develop and Validate Reporting'. Sample of Phase 3 of Info-Tech's service metric development package, 'Implement, Track, and Maintain'.
Start the development and creation of your service metrics by keeping business perspectives in mind, so they are fully aligned with business objectives. Identify the most appropriate presentation format based on stakeholder preference and need for metrics. Track goals and success metrics for your service metrics programs. It allows you to set long-term goals and track your results over time.

CIOs must actively lead the design of the service metrics program

The CIO must actively demonstrate support for the service metrics program and lead the initial discussions to determine what matters to business leaders.

  1. Lead the initiative by defining the need
    Show visible support and demonstrate importance
  2. Articulate the value to both IT and the business
    Establish the urgency and benefits
  3. Select and assemble an implementation group
    Find the best people to get the job done
  4. Drive initial metrics discussions: goals, objectives, actions
    Lead brainstorming with senior business leaders
  5. Work with the team to determine presentation formats and communication methods
    Identify the best presentation approach for senior stakeholders
  6. Establish a feedback loop for senior management
    Solicit feedback on improvements
  7. Validate the success of the metrics
    Confirm service metrics support business outcomes

Measure the success of your service metrics

It is critical to determine if the designed service metrics are fulfilling their intended purpose. The process of maintaining the service metrics program and the outcomes of implementing service metrics need to be monitored and tracked.

Validating Service Metrics Design

Target Outcome

Related Metrics

The business is enabled to identify and improve service performance to their end customer # of improvement initiatives created based on service metrics
$ cost savings/revenue generated due to actions derived from service metrics

Procedure to validate the usefulness of IT metrics

# / % of service metrics added/removed per year

Alignment between IT and business objectives and processes Business’ satisfaction with IT

Measure the success of your service metrics

It is critical to determine if the designed service metrics are fulfilling their intended purpose. The process of maintaining the service metrics program and the outcomes of implementing service metrics need to be monitored and tracked.

Validating Service Metrics Process

Target Outcome

Related Metrics

Properly defined service metrics aligned with business goals/outcomes
Easy understood measurement methodologies
% of services with (or without) defined service metrics

% of service metrics tied to business goals

Consistent approach to review and adjust metrics# of service metrics adjusted based on service reviews

% of service metrics reviewed on schedule

Demonstrate monetary value and impact through the service metrics program

In a study done by the Aberdeen Group, organizations engaged in the use of metrics benchmarking and measurement have:
  • 88% customer satisfaction rate
  • 60% service profitability
  • 15% increase in workforce productivity over the last 12 months

Stock image of a silhouette of three people's head and shoulders.
(Source: Aberdeen Group. “Service Benchmarking and Measurement.”)

A service metric is defined for: “Response time for Business Application A

The expected response time has not been achieved and this is visible in the service metrics. The reduced performance has been identified as having an impact of $250,000 per month in lost revenue potential.

The service metric drove an action to perform a root-cause analysis, which identified a network switch issue and drove a resolution action to fix the technology and architect redundancy to ensure continuity.

The fix eliminated the performance impact, allowing for recovery of the $250K per month in revenue, improved end-user confidence in the organization, and increased use of the application, creating additional revenue.

Implementing and measuring a video conferencing service

CASE STUDY
Industry: Manufacturing | Source: CIO interview and case material
Situation

The manufacturing business operates within numerous countries and requires a lot of coordination of functions and governance oversight. The company has monthly meetings, both regional and national, and key management and executives travel to attend and participate in the meetings.

Complication

While the meetings provide a lot of organizational value, the business has grown significantly and the cost of business travel has started to become prohibitive.

Action

It was decided that only a few core meetings would require onsite face-to-face meetings, and for all other meetings, the company would look at alternative means. The face-to-face aspect of the meetings was still considered critical so they focused on options to retain that aspect.

The IT organization identified that they could provide a video conferencing service to meet the business need. The initiative was approved and rolled out in the organization.

Result:

IT service metrics needed to be designed to confirm that the expected value outcome of the implementation of video conferencing was achieved.

Under the direction of the CIO, the business goals and needs driving use of the service (i.e. reduction in travel costs, efficiency, no loss of positive outcome) were used to identify success criteria and key questions to confirm success.

With this information, the service manager was able to implement relevant service metrics in business language and confirmed an 80% adoption rate and a 95% success rate in term meetings running as expected and achieving core outcomes.

Use these icons to help direct you as you navigate this research

Use these icons to help guide you through each step of the blueprint and direct you to content related to the recommended activities.

A small monochrome icon of a wrench and screwdriver creating an X.

This icon denotes a slide where a supporting Info-Tech tool or template will help you perform the activity or step associated with the slide. Refer to the supporting tool or template to get the best results and proceed to the next step of the project.

A small monochrome icon depicting a person in front of a blank slide.

This icon denotes a slide with an associated activity. The activity can be performed either as part of your project or with the support of Info-Tech team members, who will come onsite to facilitate a workshop for your organization.

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

Guided Implementation

Workshop

Consulting

"Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful." "Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track." "We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place." "Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project."

Diagnostics and consistent frameworks used throughout all four options

Develop meaningful service metrics to ensure business and user satisfaction

1. Design the Metrics 2. Design Reports and Dashboards 3. Implement, Track, and Maintain
Supporting Tool icon

Best-Practice Toolkit

  1. Defining stakeholder needs for IT based on their success criteria
  2. Derive meaningful service metrics based on identified IT services and validate with business stakeholders
  3. Validate metrics can be collected and measured
  4. Determine calculation methodology
  1. Presentation format selected based on stakeholder needs and preference for information
  2. Presentation format validated with stakeholders
  1. Identify metrics that will be presented first to the stakeholders based on urgency or impact of the IT service
  2. Determine the process to collect data, select initial targets, and integrate with SLM and BRM functions
  3. Roll out the metrics implementation for a broader audience
  4. Establish roles and timelines for metrics maintenance

Guided Implementations

  • Design metrics based on business needs
  • Validate the metrics
  • Select presentation format
  • Review metrics presentation design
  • Select and implement pilot metrics
  • Determine rollout process and establish maintenance/tracking mechanism
Associated Activity icon

Onsite Workshop

Module 1:
Derive Service Metrics From Business Goals
Module 2:
Select and Design Reports and Dashboards
Module 3:
Implement, Track, and Maintain Your Metrics to Ensure Success
Phase 1 Outcome:
  • Meaningful service metrics designed from stakeholder needs
Phase 2 Outcome:
  • Appropriate presentation format selected for each stakeholder
Phase 3 Outcome:
  • Metrics implemented and process established to maintain and track program success

Workshop overview

Contact your account representative or email Workshops@InfoTech.com for more information.
Workshop Day 1 Workshop Day 2 Workshop Day 3 Workshop Day 4
Design the Metrics
Determine Presentation Format and Implement Metrics
Gather Service Level Requirements
Monitor and Improve Service Levels

Activities

  • 1.1 Determine stakeholder needs
  • 1.2 Determine success criteria and key performance indicators
  • 1.3 Derive metrics
  • 1.4 Validate the metric collection
  • 2.1 Discuss stakeholder needs/preference for data and select presentation format
  • 2.2 Select and design the metric report
  • Requirements
  • 3.1 Determine the business requirements
  • 3.2 Negotiate service levels
  • 3.3 Align operational level agreements (OLAs) and supplier contracts
  • 4.1 Conduct service report and perform service review
  • 4.2 Communicate service review
  • 4.3 Remediate issues using action plan
  • 4.4 Proactive prevention

Deliverables

  1. Metrics Development Workbook
  1. Metrics Presentation Format Selection Guide
  2. Metrics Tracking Tool
  1. Service Level Management SOP
  2. Service Level Agreement
  1. Service Level Report
  2. Service Level Review
  3. Business Satisfaction Report

Develop Meaningful Service Metrics to Ensure Business and User Satisfaction

PHASE 1

Design the Metrics

Step (1): Design the Metrics

PHASE 1 PHASE 2 PHASE 3

1.1

Derive the Service Metrics

1.2

Validate the Metrics

2.1

Determine Reporting Format

3.1

Select Pilot Metrics

3.2

Activate and Maintain Metrics

This step involves the following participants:

  • CIO
  • Business Relationship Manager (BRM)
  • Service Level Manager (SLM)

Outcomes of this step

  • Defined stakeholder needs for IT based on their success criteria
  • Identified IT services that are tied to the delivery of business outcomes
  • Derived meaningful service metrics based on identified IT services and validated with business stakeholders
  • Validated that metrics can be collected and measured
  • Determined calculation methodology

Phase 1 outline

Associated Activity icon Call 1-888-670-8889 or email GuidedImplementations@InfoTech.com for more information.

Complete these steps on your own, or call us to complete a guided implementation. A guided implementation is a series of 2-3 advisory calls that help you execute each phase of a project. They are included in most advisory memberships.

Guided Implementation 1: Design the Metrics

Proposed Time to Completion (in weeks): 4 weeks
Step 1.1: Design Metrics Step 1.2: Validate the Metrics
Start with an analyst kick-off call:
  • Determine the stakeholder and their needs
  • Identify IT services that are tied to the delivery of business outcomes
  • Derive the service metrics
Review findings with analyst:
  • For the selected metrics, identify the data source for collection
  • Validate whether or not the data can be created
  • Create a calculation method for the metrics
Then complete these activities…
  • Using the methodology provided, identify additional stakeholders and map out their success criteria, including KPIs to determine the appropriate service metrics
Then complete these activities…
  • Determine whether the designed metrics are measurable, and if so, how
With these tools & templates:
  • Metrics Development Workbook
With these tools & templates:
  • Metrics Development Workbook

Design your service metrics – overview

Figure representing 'CIO'. Step 1
Derive your service metrics

Metrics Worksheet

Figure representing 'SLM' and/or 'BRM'. Step 2
Validate your metrics

Metrics Worksheet

Figures representing 'CIO', 'SLM', and/or 'BRM'. Step 3
Confirm with stakeholders

Metrics Tracking Sheet

A star.

Defined IT Service Metrics

Deriving the right metrics is critical to ensuring that you will generate valuable and actionable service metrics.

Derive your service metrics from business objectives and needs

Service metrics must be designed with the business perspective in mind so they are fully aligned with business objectives.

Thus, IT must start by identifying specific stakeholder needs. The more IT understands about the business, the more relevant the metrics will be to the business stakeholders.

  1. Who are your stakeholders?
  2. What are their goals and pain points?
  3. What do the stakeholders need to know?
  4. What do I need to measure?
  5. Derive your service metrics

Derive your service metrics

Supporting Tool icon 1.1 Metrics Development Workbook

This workbook guides the development and creation of service metrics that are directly tied to stakeholder needs.

This process will ensure that your service metrics are designed with the business perspective in mind so they are fully aligned with business objectives.

  1. Who are the relevant stakeholders?
  2. What are the goals and pain points of your stakeholders?
  3. What do the stakeholders need to know?
  4. What does IT need to measure?
  5. What are the appropriate IT metrics?

Download the Metrics Development Workbook.

Sample of Info-Tech's Metrics Development Workbook.

Determine your stakeholders

Supporting Tool icon 1.1 0.5 Hour

Who are your stakeholders?

  1. Identify the primary stakeholders of your service metrics. Stakeholders are the people who have a very specific need to know about how IT services affect their business outcomes. Different stakeholders can have different perspective on the same IT service metric.Most often, the primary target of service metrics are the business stakeholders, e.g. VP of a business unit.
  2. Identify any additional stakeholders. The CIO is also a stakeholder since they are effectively the business relationship manager for the senior leaders.

Video Conferencing Case Study
Manufacturing company

For this phase, we will demonstrate how to derive the service metrics by going through the steps in the methodology.

At a manufacturing company, the CIO’s main stakeholder is the CEO, whose chief concern is to improve the financial position of the company.

Identify goals and pain points of your stakeholders

Supporting Tool icon 1.2 0.5 Hour

What are their goals and pain points?

  1. Clearly identify each stakeholder’s business goals and outcomes. These would be particular business goals related to a specific business unit.
  2. Identify particular pain points for each business unit to understand what is preventing them from achieving the desirable business outcome.

VC Case Study

One of the top initiatives identified by the company to improve financial performance was to reduce expense.

Because the company has several key locations in different states, company executives used to travel extensively to carry out meetings at each location.

Therefore, travel expenses represent a significant proportion of operational expenses and reducing travel costs is a key goal for the company’s executives.

What do the stakeholders need to know?

Supporting Tool icon 1.3 0.5 Hour

What do the stakeholders need to know?

  1. Identify the key things that the stakeholders would need to know based on the goals and pain points derived from the previous step.These are your success criteria and must be met to successfully achieve the desired goals.

VC Case Study

The CEO needs to have assurance that without executives traveling to each location, remote meetings can be as effective as in-person meetings.

These meetings must provide the same outcome and allow executives to collaborate and make similar strategic decisions without the onsite, physical presence.

Therefore, the success criteria are:

  • Reduced travel costs
  • Effective collaboration
  • High-quality meetings

What do I need to measure?

Supporting Tool icon 1.4 1 Hour

What does IT need to measure?

  1. Identify the IT services that are leveraged to achieve the business goals and success criteria.
  2. Identify the users of those services and determine the nature of usage for each group of users.
  3. Identify the key indicators that must be measured for those services from an IT perspective.

VC Case Study

The IT department decides to implement the video conferencing service to reduce the number of onsite meetings. This technology would allow executives to meet remotely with both audio and video and is the best option to replicate a physical meeting.

The service is initially available to senior executives and will be rolled out to all internal users once the initial implementation is deemed successful.

To determine the success of the service, the following needs to be measured:

  1. Outcomes of VC meetings
  2. Quality of the VC meetings
  3. Reduction in travel expenses

Derive service metrics

Supporting Tool icon 1.5 0.5 Hour

Derive your service metrics

  1. Derive the service metrics that are meaningful to business stakeholders based on the IT services and the key indicators identified in the previous steps.
  2. Distinguish between service metrics and business metrics. You may identify some business metrics in addition to the IT metrics, and although these are important, IT doesn’t own the process of tracking and reporting business metrics.

VC Case Study

In the previous step, IT identified that it must measure the outcomes of VC meetings, quality of the VC meetings, and the reduction in travel expenses. From these, the appropriate service metrics can be derived to answer the needs of the CEO.

IT needs to measure:

  1. Percent of VC meetings successfully delivered
  2. Growth of number of executive meetings conducted via VC
Outcomes

IT also identified the following business metrics:

  1. Reduction in percent of travel expense/spend
  2. Reduction in lost time due to travel

Validate your metrics

Once appropriate service metrics are derived from business objectives, the next step is to determine whether or not it is viable to actually measure the metrics.

Can you measure it? The first question IT must answer is whether the metric is measurable. IT must identify the data source, validate its ability to collect the data, and specify the data requirement. Not all metrics can be measured!
How will you measure it? If the metric is measurable, the next step is to create a way to measure the actual data. In most cases, simple formulas that can be easily understood are the best approach.
Define your actions Metrics must be used to drive or reinforce desirable outcomes and behaviors. Thus, IT must predetermine the necessary actions associated with the different metric levels, thresholds, or trends.

Determine if you can measure the identified metric

Supporting Tool icon 1.6 0.5 Hour

INSTRUCTIONS

  1. Determine what data sources are available. Make sure that you know where the information you need is captured, or will need to be captured. This would include:
    • A ticket/request system
    • An auto discovery tool
    • A configuration management database ( CMDB)
  2. Confirm that IT has the ability to collect the information.
    • If the necessary data is already contained in an identified data source, then you can proceed.
    • If not, consider whether it’s possible to gather the information using current sources and systems.
    • Understand the constraints and cost/ROI to implement new technology or revise processes and data gathering to produce the data.

VC Case Study

Using the metric derived from the video conferencing service example, IT wants to measure the % of VC meetings successfully delivered.

What are the data sources?

  • Number of VC meetings that took place
  • Number of service incidents
  • User survey

Determine if you can measure the identified metric

Supporting Tool icon 1.6 0.5 Hour

INSTRUCTIONS

  1. Understand your data requirements
    • To produce relevant metrics from your data, you need to ensure the level of quality and currency that provides you with useful information. You need to define:
      • The level of detail that has to be captured to make the data useful.
      • The consistency of the data, and how it needs to be entered or gathered.
      • The accuracy of the data. This includes how current the data needs to be, how quickly changes have to be made, and how data quality will be verified.

VC Case Study

Data requirement for percent of successful VC meetings:

  • Level of detail – user category, location, date/time,
  • Consistency – how efficiently are VC-related incidents opened and closed? Is the data collected and stored consistently?
  • Accuracy – is the information entered accurately?

Create the calculation to measure it

Supporting Tool icon 1.7 0.5 Hour

Determine how to calculate the metrics.

INSTRUCTIONS
  1. Develop the calculations that will be used for each accepted metric. The measurement needs to be clear and straightforward.
  2. Define the scope and assumptions for each calculation, including:
    • The defined measurement period (e.g. monthly, weekly)
    • Exclusions (e.g. nonbusiness hours, during maintenance windows)

VC Case Study

Metric: Percent of VC meetings delivered successfully

IT is able to determine the total number of VC meetings that took place and the number of VC service requests to the help desk.

That makes it possible to use the following formula to determine the success percentage of the VC service:

((total # VC) – (# of VC with identified incidents)) / (total # VC) * 100

Define the actions to be taken for each metric

Supporting Tool icon 1.7 1.5 Hour

INSTRUCTIONS

Centered on the defined metrics and their calculations, IT can decide on the actions that should be driven out of each metric based on one of the following scenarios:
  • Scenario 1: Ad hoc remedial action and root-cause investigation. If the reason for the result is unknown, determining root cause or identifying trends is required to determine required actions.
  • Scenario 2: Predefined remedial action. A set of predetermined actions associated with different results. This is useful when the meaning of the results is clear and points to specific issues within the environment.
  • Scenario 3: Nonremedial action. The metrics may produce a result that reinforces or supports company direction and strategy, or identifies an opportunity that may drive a new initiative or idea.

VC Case Study

If the success rate of the VC meetings is below 90%, IT needs to focus on determining if there is a common cause and identify if this is a consistent downward trend.

A root-cause analysis is performed that identifies that network issues are causing difficulties, impacting the connection quality and usability of the VC service.

Validate the confirmed metrics with the business

Supporting Tool icon 1.8 1 Hour

INPUT: Selected service metrics, Discussion with the business

OUTPUT: Validated metrics with the business

Materials: Metrics with calculation methodology

Participants: IT and business stakeholders, Service owners

INSTRUCTIONS

  1. Once you have derived the appropriate metrics and established that the metrics are measurable, you must go back to the targeted stakeholders and validate that the selected metrics will provide the right information to meet their identified goals and success criteria.
  2. Add confirmed metrics to the Metrics Tracking Tool, in the Metrics Tracking Plan tab.
Service Metric Corresponding
Business Goal
Measurement
Method
Defined Actions

Example: Measuring the online banking service at a financial institution

Who are IT’s stakeholders? The financial institution provides various banking solutions to its customers. Retail banking is a core service offered by the bank and the VP of retail banking is a major stakeholder of IT.
What are their goals and pain points? The VP of retail banking’s highest priorities are to increase revenue, increase market share, and maintain the bank’s brand and reputation amongst its customers.
What do they need to know? In order to measure success, the VP of retail banking needs to determine performance in attracting new clients, retaining clients, expanding into new territory, and whether they have increased the number of services provided to existing clients.
What does IT need to measure? The recent implementation of an online banking service is a key initiative that will keep the bank competitive and help retail banking meet its goals. The key indicators of this service are: the total number of clients, the number of products per client, percent of clients using online banking, number of clients by segment, service, territory.
Derive the service metrics Based on the key indicators, IT can derive the following service metrics:
1. Number of product applications originated from online banking
2. Customer satisfaction/complaints
As part of the process, IT also identified some business metrics, such as the number of online banking users per month or the number of times a client accesses online banking per month.

Design service metrics to track service performance and value

CASE STUDY
Industry: Manufacturing | Source: CIO
Challenge Solution Results
The IT organization needed to generate metrics to show the business whether the video conferencing service was being adopted and if it was providing the expected outcome and value.

Standard IT metrics were technical and did not provide a business context that allowed for easy understanding of performance and decision making.

The IT organization, working through the CIO and service managers, sat down with the key business stakeholders of the video conferencing service.

They discussed the goals for the meeting and defined the success criteria for those goals in the context of video conference meeting outcomes.

The success criteria that were discussed were then translated into a set of questions (key performance indicators) that if answered, would show that the success criteria were achieved.

The service manager identified what could be measured to answer the defined questions and eliminated any metrics that were either business metrics or non-IT related.

The remaining metrics were identified as the possible service metrics, and the ability to gather the information and produce the metric was confirmed.

Service metrics were defined for:

  1. Percent of video conference meetings delivered successfully
  2. Growth in the number of executive meetings conducted via video conference

If you want additional support, have our analysts guide you through this phase as part of an Info-Tech Workshop Associated Activity icon

Book a workshop with our Info-Tech analysts:

Photo of Valence Howden, Senior Manager, CIO Advisory, Info-Tech Research Group.
  • To accelerate this project, engage your IT team in an Info-Tech workshop with an Info-Tech analyst team.
  • Info-Tech analyst will join you and your team onsite at your location or welcome you to Info-Tech's historic Toronto office to participate in an innovative onsite workshop.
  • Contact your account manager (www.infotech.com/account), or email Workshops@InfoTech.com for more information.

The following are sample activities that will be conducted by Info-Tech analysts with your team:

1.1

Sample of activity 1.1 'Determine your stakeholders'. Determine stakeholder needs, goals, and pain points

The onsite analyst will help you select key stakeholders and analyze their business objectives and current pain points.

1.2

Sample of activity 1.2 'Identify goals and pain points of your stakeholders'. Determine the success criteria and related IT services

The analyst will facilitate a discussion to uncover the information that these stakeholders care about. The group will also identify the IT services that are supporting these objectives.

If you want additional support, have our analysts guide you through this phase as part of an Info-Tech Workshop Associated Activity icon

Book a workshop with our Info-Tech analysts:

1.5

Sample of activity 1.5 'Derive service metrics'. Derive the service metrics

Based on the key performance indicators obtained in the previous page, derive meaningful business metrics that are relevant to the stakeholders.

1.6

Sample of activity 1.6 'Determine if you can measure the identified metric'. Validate the data collection process

The analyst will help the workshop group determine whether the identified metrics can be collected and measured. If so, a calculation methodology is created.

1.7

Sample of activity 1.7 'Create the caluclation to measure it'. Validate metrics with stakeholders

Establish a feedback mechanism to have business stakeholders validate the meaningfulness of the metrics.

Develop Meaningful Service Metrics to Ensure Business and User Satisfaction

PHASE 2

Design Reports and Dashboards

Step (2): Design Reports and Dashboards

PHASE 1PHASE 2PHASE 3

1.1

Derive the Service Metrics

1.2

Validate the Metrics

2.1

Determine Reporting Format

3.1

Select Pilot Metrics

3.2

Activate and Maintain Metrics

This step involves the following participants:

  • Business Relationship Manager
  • Service Level Manager
  • Business Stakeholders

Outcomes of this step

  • Presentation format selected based on stakeholder needs and preference for information
  • Presentation format validated with stakeholders

Phase 2 outline

Associated Activity icon Call 1-888-670-8889 or email GuidedImplementations@InfoTech.com for more information.

Complete these steps on your own, or call us to complete a guided implementation. A guided implementation is a series of 2-3 advisory calls that help you execute each phase of a project. They are included in most advisory memberships.

Guided Implementation 2: Design Reports and Dashboards

Proposed Time to Completion (in weeks): 3 weeks
Step 2.1: Select Presentation Format Step 2.2: Review Design
Start with an analyst kick-off call:
  • Review the different format of metrics presentation and discuss the pros/cons of each format
  • Discuss stakeholder needs/preference for data
  • Select the presentation format
Review findings with analyst:
  • Discuss stakeholder feedback based on selected presentation format
  • Modify and adjust the presentation format as needed
Then complete these activities…
  • Design the metrics using the selected format
Then complete these activities…
  • Finalize the design for metrics presentation
With these tools & templates:
  • Metrics Presentation Format Selection Guide
With these tools & templates:
  • Metrics Presentation Format Selection Guide

Design the reports – overview

Figure representing 'SLM' and/or 'BRM'. Step 1
Understand the pros and cons of different reporting styles
Figure representing 'SLM' and/or 'BRM'. Step 2
Determine your reporting and presentation style

Presentation Format Selection

Figure representing 'SLM' and/or 'BRM'. Step 3
Design your metrics reports
A star.

Validated Service Reports

The design of service metrics reporting is critically important. The reporting style must present the right information in the most interesting and stakeholder-centric way possible to ensure that it is read and used.

The reports must also display information in a way that generates actions. If your stakeholders cannot make decisions, kick off activities, or ask questions based on your reports, then they have no value.

Determine the right presentation format for your metrics

Most often, metrics are presented in the following ways:

Dashboard
(PwC. “Mega-Trends and Implications.”)
Sample of the 'Dashboard' metric presentation format.
Infographic
(PwC. “Healthcare’s new entrants.”)
Sample of the 'Infographic' metric presentation format.
Report
(PwC Blogs. “Northern Lights.”)
Sample of the 'Report' metric presentation format.
Scorecard
(PwC. “Annual Report 2015.”)
Sample of the 'Scorecard' metric presentation format.

Understand the advantages and disadvantages of each reporting style – Dashboard

A dashboard is a reporting method that provides a dynamic at-a-glance view of key metrics from the perspective of key stakeholders. It provides a quick graphical way to process important performance information in real time.

Features

Typically web-based

Dynamic data that is updated in real time

Advantage

Aggregates a lot of information into a single view

Presents metrics in a simplistic style that is well understood

Provides a quick point-in-time view of performance

Easy to consume visual presentation style

Disadvantage

Complicated to set up well.
Requires additional technology support: programming, API, etc.

Promotes a short-term outlook – focus on now, no historical performance and no future trends. Doesn’t provide the whole picture and story.

Existing dashboard tools are often not customized enough to provide real value to each stakeholder.

Dashboards present real-time metrics that can be accessed and viewed at any time

Sample of the 'Dashboard' metric presentation format.
(Source: PwC. “Mega-Trends and Implications.”)
Metrics presented through online dashboards are calculated in real time, which allows for a dynamic, current view into the performance of IT services at any time.

Understand the advantages and disadvantages of each reporting style – Infographic

An infographic is a graphical representation of metrics or data, which is used to show information quickly and clearly. It’s based on the understanding that people retain and process visual information more readily than written details.

Features

Turns dry into attractive –transforms data into eye-catching visual memory that is easier to retain

Can be used as the intro to a formal report

There are endless types of infographics

Advantage

Easily consumable

Easy to retain

Eye catching

Easily shared

Spurs conversation

Customizable

Disadvantage

Require design expertise and resources

Can be time consuming to generate

Could be easily misinterpreted

Message can be lost with poor design

Infographics allow for completely unique designs

Sample of the 'Infographic' metric presentation format.
(Source: PwC. “Healthcare’s new entrants…”)
There is no limit when it comes to designing an infographic. The image used here visually articulates the effects of new entrants pulling away the market.

Understand the advantages and disadvantages of each reporting style – Formal Report

A formal report is a more structured and official reporting style that contains detailed research, data, and information required to enable specific business decisions, and to help evaluate performance over a defined period of time.

Definition

Metrics can be presented as a component of a periodic, formal report

A physical document that presents detailed information to a particular audience

Advantage

More detailed, more structured and broader reporting period

Formal, shows IT has put in the effort

Effectively presents a broader and more complete story

Targets different stakeholders at the same time

Disadvantage

Requires significant effort and resources

Higher risk if the report does not meet the expectation of the business stakeholder

Done at a specific time and only valuable for that specific time period

Harder to change format

Formal reports provide a detailed view and analysis of performance

Sample of the 'Formal Report' metric presentation format.
(Source: PwC Blogs. “Northern Lights: Where are we now?”)
An effective report incorporates visuals to demonstrate key improvements.

Formal reports can still contain visuals, but they are accompanied with detailed explanations.

Understand the advantages and disadvantages of each reporting style – Scorecard

A scorecard is a graphic view of the progress and performance over time of key performance metrics. These are in relation to specified goals based on identified critical stakeholder objectives.

Features

Incorporates multiple metrics effectively.

Scores services against the most important organizational goals and objectives. Scorecards may tie back into strategy and different perspectives of success.

Advantage

Quick view of performance against objectives

Measure against a set of consistent objectives

Easily consumable

Easy to retain

Disadvantage

Requires a lot of forethought

Scorecards provide a time-bound summary of performance against defined goals

Sample of the 'Scorecard' metric presentation format.
(PwC. “Annual Report 2015.”)
Scorecards provide a summary of performance that is directly linked to the organizational KPIs.

Determine your report style

Supporting Tool icon 2.1 Metrics Presentation Format Selection Guide

In this section, you will determine the optimal reporting style for the service metrics.

This guide contains four questions, which will help IT organizations identify the most appropriate presentation format based on stakeholder preference and needs for metrics.

  1. Who is the relevant stakeholder?
  2. What are the defined actions for the metric?
  3. How frequently does the stakeholder need to see the metric?
  4. How does the stakeholder like to receive information?
Sample of Info-Tech's Metrics Presentation Format Selection Guide.
Download the Metrics Presentation Format Selection Guide.

Determine your best presentation option

Supporting Tool icon 2.1 2 Hours

INPUT: Identified stakeholder and his/her role

OUTPUT: Proper presentation format based on need for information

Materials: Metrics Presentation Format Selection Guide

Participants: BRM, SLM, Program Manager

After deciding on the report type to be used to present the metric, the organization needs to consider how stakeholders will consume the metric.

There are three options based on stakeholder needs and available presentation options within IT.

  1. Paper-based presentation is the most traditional form of reporting and works well with stakeholders who prefer physical copies. The report is produced at a specific time and requires no additional IT capability.
  2. Online documents stored on webpages, SharePoint, or another knowledge management system could be used to present the metrics. This allows the report to be linked to other information and easily shared.
  3. Online dashboards and graphics can be used to have dynamic, real-time reporting and anytime access. These webpages can be incorporated into an intranet and allow the user to view the metrics at any time. This will require IT to continuously update the data in order to maintain the accuracy of the metrics.

Design your metric reports with these guidelines in mind

Supporting Tool icon 2.2 30 Minutes
  1. Stakeholder-specificThe report must be driven by the identified stakeholder needs and preferences and articulate the metrics that are important to them.
  2. ClarityTo enable decision making and drive desired actions, the metrics must be clear and straightforward. They must be presented in a way that clearly links the performance measurement to the defined outcome without leading to different interpretations of the results.
  3. SimplicityThe report must be simple to read, understand, and analyze. The language of the report must be business-centric and remove as much complexity as possible in wording, imaging, and context.

Be sure to consider access rights for more senior reports. Site and user access permissions may need to be defined based on the level of reporting.

Metrics reporting on the video conferencing service

CASE STUDY
Industry: Manufacturing | Source: CIO Interview
The Situation

The business had a clear need to understand if the implementation of video conferencing would allow previously onsite meetings to achieve the same level of effectiveness.

Reporting Context

Provided reports had always been generated from an IT perspective and the business rarely used the information to make decisions.

The metrics needed to help the business understand if the meetings were remaining effective and be tied into the financial reporting against travel expenses, but there would be limited visibility during the executive meetings.

Approach

The service manager reviewed the information that he had gathered to confirm how often they needed information related to the service. He also met with the CIO to get some insight into the reports that were already being provided to the business, including the ones that were most effective.

Considerations

The conversations identified that there was no need for a dynamic real-time view of the performance of the service, since tracking of cost savings and utility would be viewed monthly and quarterly. They also identified that the item would be discussed within a very small window of time during the management meetings.

The Solution

It was determined that the best style of reporting for the metric was an existing scorecard that was produced monthly, using some infographics to ensure that the information is clear at a glance to enable quick decision making.

If you want additional support, have our analysts guide you through this phase as part of an Info-Tech Workshop Associated Activity icon

Book a workshop with our Info-Tech analysts:

Photo of Valence Howden, Senior Manager, CIO Advisory, Info-Tech Research Group.
  • To accelerate this project, engage your IT team in an Info-Tech workshop with an Info-Tech analyst team.
  • Info-Tech analyst will join you and your team onsite at your location or welcome you to Info-Tech's historic Toronto office to participate in an innovative onsite workshop.
  • Contact your account manager (www.infotech.com/account), or email Workshops@InfoTech.com for more information.

The following are sample activities that will be conducted by Info-Tech analysts with your team:

2.1

Sample of presentation format option slide 'Determine the right presentation format for your metrics'. Understand the different presentation options

The onsite analyst will introduce the group to the communication vehicles of infographic, scorecard, formal report, and dashboard.

2.1

Sample of activity 2.1 'Determine your best presentation option'. Assess stakeholder needs for information

For selected stakeholders, the analyst will facilitate a discussion on how stakeholders would like to view information and how the metrics can be presented to aid decision making.

If you want additional support, have our analysts guide you through this phase as part of an Info-Tech Workshop Associated Activity icon

Book a workshop with our Info-Tech analysts:

2.2

Sample of activity 2.2 'Design your metric reports with these guidelines in mind'. Select and design the metric report

Based on the discussion, the working group will select the most appropriate presentation format and create a rough draft of how the report should look.

Develop Meaningful Service Metrics to Ensure Business and User Satisfaction

PHASE 3

Implement, Track, and Maintain Your Metrics

Step (3): Implement, Track, and Maintain Your Metrics

PHASE 1PHASE 2PHASE 3

1.1

Derive the Service Metrics

1.2

Validate the Metrics

2.1

Determine Reporting Format

3.1

Select Pilot Metrics

3.2

Activate and Maintain Metrics

This step involves the following participants:

  • Service Level Manager
  • Business Relationship Manager
  • Service Metrics Program Manager

Activities in this step

  • Determine the first batch of metrics to be implemented as part of the pilot program
  • Create a process to collect and validate data, determine initial targets, and integrate with SLM and BRM functions
  • Present the metric reports to the relevant stakeholders and incorporate the feedback into the metric design
  • Establish a standard process and roll out the implementation of metrics in batches
  • Establish a process to monitor and track the effectiveness of the service metrics program and make adjustments when necessary

Phase 3 outline

Associated Activity icon Call 1-888-670-8889 or email GuidedImplementations@InfoTech.com for more information.

Complete these steps on your own, or call us to complete a guided implementation. A guided implementation is a series of 2-3 advisory calls that help you execute each phase of a project. They are included in most advisory memberships.

Guided Implementation 3: Implement, Track, and Maintain Your Metrics

Proposed Time to Completion (in weeks): 4 weeks
Step 3.1: Select and Launch Pilot Metrics Step 3.2: Track and Maintain the Metrics
Start with an analyst kick-off call:
  • Identify metrics that will be presented first to the stakeholders based on urgency or impact of the IT service
  • Determine the process to collect data, select initial targets, and integrate with SLM and BRM functions
Review findings with analyst:
  • Review the success of metrics and discuss feedback from stakeholders
  • Roll out the metrics implementation to a broader audience
  • Establish roles and timelines for metrics maintenance
Then complete these activities…
  • Document the first batch of metrics
  • Document the baseline, initial targets
  • Create a plan to integrate with SLM and BRM functions
Then complete these activities…
  • Create a document that defines how the organization will track and maintain the success of the metrics program
  • Review the metrics program periodically
With these tools & templates:
  • Metrics Tracking Tool
With these tools & templates:
  • Metrics Tracking Tool

Implement, Track, and Maintain the Metrics

Figure representing 'SLM' and/or 'BRM'. Step 1
Run your pilot

Metrics Tracking Tool

Figure representing 'SLM' and/or 'BRM'. Step 2
Validate success

Metrics Tracking Tool

Figure representing 'SLM' and/or 'BRM'. Step 3
Implement your metrics program in batches

Metrics Tracking Tool

A star.

Active Service Metrics Program

Once you have defined the way that you will present the metrics, you are ready to run a pilot with a smaller sample of defined service metrics.

This allows you to validate your approach and make refinements to the implementation and maintenance processes where necessary, prior to activating all service metrics.

Track the performance of your service metrics

Supporting Tool icon 3.1

The Metrics Tracking Tool will enable you to track goals and success metrics for your service metrics programs. It allows you to set long-term goals and track your results over time.

There are three sections in this tool:
  1. Metrics Tracking Plan. Identify the metrics to be tracked and their purpose.
  2. Metrics Tracking Actuals. Monitor and track the actual performance of the metrics.
  3. Remediation Tracking. Determine and document the steps that need to be taken to correct a sub-performing metric.
Sample of Info-Tech's Metrics Tracking Tool.

Select pilot metrics

Supporting Tool icon 3.1 30 Minutes

INPUT: Identified services, Business feedback

OUTPUT: Services with most urgent need or impact

Materials: Service catalog or list of identified services

Participants: BRM, SLM, Business representatives

To start the implementation of your service metrics program and drive wider adoption, you need to run a pilot using a smaller subset of metrics.

INSTRUCTIONS

To determine the sample for the pilot, consider metrics that:

  • Are related to critical business services and functions
  • or
  • Address known/visible pain points for the business
  • or
  • Were designed for supportive or influential stakeholders

Metrics that meet two or more criteria are ideal for the pilot

Collect and validate data

Supporting Tool icon 3.2 1 Hour

INPUT: Identified metrics

OUTPUT: A data collection mythology, Metrics tracking

Materials: Metrics

Participants: SLM, BRM, Service owner

You will need to start collection and validation of your identified data in order to calculate the results for your pilot metrics.

INSTRUCTIONS

  1. Initiate data collection
    • Use the data sources identified during the design phase and initiate the data collection process.
  2. Determine start date
    • If historical data can be retrieved and gathered, determine how far back you want your measurements to start.
  3. Compile data and validate
    • Ensure that the information is accurate and up to date. This will require some level of data validation and audit.
  4. Run the metric
    • Use the defined calculation and source data to generate the metrics result.
  5. Record metrics results
    • Use the metrics tracking sheet to track the actual results.

Determine initial targets

Supporting Tool icon 3.3 1 Hour

INPUT: Historical data/baseline data

OUTPUT: Realistic initial target for improvement

Materials: Metrics Tracking Tool

Participants: BRM, SLM, Service owner

INSTRUCTIONS

Identify an initial service objective based on one or more of the following options:

  1. Establish an initial target using historical data and trends of performance.
  2. Establish an initial target based on stakeholder-identified requirements and expectations.
  3. Run the metrics report over a defined period of time and use the baseline level of achievement to establish an initial target.

The target may not always be a number - it could be a trend. The initial target will be changed after review with stakeholders

Integrate with SLM and BRM processes

Supporting Tool icon 3.4 1 Hour

INPUT: SLM and BRM SOPs or responsibility documentations

OUTPUT: Integrate service metrics into the SLM/BRM role

Materials: SLM / BRM reports

Participants: SLM, BRM, CIO, Program manager, Service manager

The service metrics program is usually initiated, used, and maintained by the SLM and BRM functions.

INSTRUCTIONS

Ensure that the metrics pilot is integrated with those functions by:

  1. Engaging with SLM and BRM functions/resources
    • Identify SLM and BRM resources associated with or working on the services where the metrics are being piloted
    • Obtain their feedback on the metrics/reporting
  2. Integrating with the existing reporting and meeting cycles
    • Ensure the metrics will be calculated and available for discussion at standing meetings and with existing reports
  3. Establishing the metrics review and validation cycle for these metrics
    • Confirm the review and validation period for the metrics in order to ensure they remain valuable and actionable

Generate reports and present to stakeholders

Supporting Tool icon 3.5 1 Hour

INPUT: Identified metrics, Selected presentation format

OUTPUT: Metrics reports that are ready for distribution

Materials: Metrics Presentation Format Selection Guide

Participants: BRM, SLM, CIO, Business representatives

INSTRUCTIONS

Once you have completed the calculation for the pilot metrics:

  1. Confirm the report style for the selected metrics (as defined in Phase 2)
  2. Generate the reporting for the pilot metrics
  3. Present the pilot metric reports to the identified BRM and SLM resources who will present the reporting to the stakeholders
  4. Gather feedback from Stakeholders on metrics - results and process
  5. Create and execute remediation plans for any actions identified from the metrics
  6. Initiate the review cycle for metrics (to ensure they retain value)

Plan the rollout and implementation of the metrics reporting program

Supporting Tool icon 3.6 1 Hour

INPUT: Feedback from pilot, Services in batch

OUTPUT: Systematic implementation of metrics

Materials: Metrics Tracking Tool

Participants: BRM, SLM, Program manager

Upon completion of the pilot, move to start the broader implementation of metrics across the organization:

INSTRUCTIONS

  1. Identify the service metrics that you will implement. They can be selected based on multiple criteria, including:
    • Organizational area/business unit
    • Service criticality
    • Pain points
    • Stakeholder engagement (detractors, supporters)
  2. Create a rollout plan for implementation in batches, identifying expected launch timelines, owners, targeted stakeholders, and communications plans
  3. Use the implementation plan from the pilot to roll out each batch of service metrics:
    • Collect and validate data
    • Determine target(s)
    • Integrate with BRM and SLM
    • Generate and communicate reports to stakeholders

Maintain the service metrics

Supporting Tool icon 3.7 1.5 Hour

INPUT: Feedback from business stakeholders

OUTPUT: Modification to individual metrics or to the process

Materials: Metrics Tracking Tool, Metrics Development Workbook

Participants: CIO, BRM, SLM, Program manager, Service owner

Once service metrics and reporting become active, it is necessary to determine the review time frame for your metrics to ensure they remain useful.

INSTRUCTIONS

  1. Confirm and establish a review time frame with stakeholders (e.g. annually, bi-annually, after organizational or strategic changes).
  2. Meet with stakeholders by the review date to discuss the value of existing metrics and validate:
    • Whether the goals associated with the metrics are still valid
    • If the metric is still necessary
    • If there is a more effective way to present the metrics
  3. Track actions based on review outcomes and update the remediation tracking sheet.
  4. Update tracking sheet with last complete review date.

Maintain the metrics

Supporting Tool icon 3.7

Based on the outcome of the review meeting, decide what needs to be done for each metric, using the following options:

Add

A new metric is required or an existing metric needs large-scale changes (example: calculation method or scope).
Triggers metrics design as shown in phases 1 and 2.

Change

A minor change is required to the presentation format or data. Note: a major change in a metric would be performed through the Add option.

Remove

The metric is no longer required, and it needs to be removed from reporting and data gathering. A final report date for that metric should be determined.

Maintain

The metric is still useful and no changes are required to the metric, its measurement, or how it’s reported.

Ensuring metrics remain valuable

VC CASE STUDY
Industry: Manufacturing | Source: CIO Interview

Reviewing the value of active metrics

When the video conferencing service was initially implemented, it was performed as a pilot with a group of executives, and then expanded for use throughout the company. It was understood that prior to seeing the full benefit in cost reduction and increased efficiency and effectiveness, the rate of use and adoption had to be understood.

The primary service metrics created for the service were based on tracking the number of requests for video conference meetings that were received by the IT organization. This identified the growth in use and could be used in conjunction with financial metrics related to travel to help identify the impact of the service through its growth phase.

Once the service was adopted, this metric continued to be tracked but no longer showed growth or expanded adoption.

The service manager was no longer sure this needed to be tracked.

Key Activity

The metrics around requests for video conference meetings were reviewed at the annual metrics review meeting with the business. The service manager asked if the need for the metric, the goal of tracking adoption, was still important for the business.

The discussion identified that the adoption rate was over 80%, higher than anticipated, and that there was no value in continuing to track this metric.

Based on the discussion, the adoption metrics were discontinued and removed from data gathering and reporting, while a success rate metric was added (how many meetings ran successfully and without issue) to ensure the ongoing value of the video conferencing service.

If you want additional support, have our analysts guide you through this phase as part of an Info-Tech Workshop Associated Activity icon

Book a workshop with our Info-Tech analysts:

Photo of Valence Howden, Senior Manager, CIO Advisory, Info-Tech Research Group.
  • To accelerate this project, engage your IT team in an Info-Tech workshop with an Info-Tech analyst team.
  • Info-Tech analyst will join you and your team onsite at your location or welcome you to Info-Tech's historic Toronto office to participate in an innovative onsite workshop.
  • Contact your account manager (www.infotech.com/account), or email Workshops@InfoTech.com for more information.

The following are sample activities that will be conducted by Info-Tech analysts with your team:

3.1

Sample of activity 3.1 'Select pilot metrics'. Select the pilot metrics

The onsite analyst will help the workshop group select the metrics that should be first implemented based on the urgency and impact of these metrics.

3.2

Sample of activity 3.2 'Collect and validate data'. Gather data and set initial targets

The analyst will help the group create a process to gather data, measure baselines, and set initial targets.

If you want additional support, have our analysts guide you through this phase as part of an Info-Tech Workshop Associated Activity icon

Book a workshop with our Info-Tech analysts:

3.5

Sample of activity 3.5 'Generate reports and present to stakeholders'. Generate the reports and validate with stakeholders

The Info-Tech analyst will help the group establish a process to receive feedback from the business stakeholders once the report is generated.

3.6

Sample of activity 3.6 'Plan the rollout and implementation of the metrics reporting program'. Implement the service metrics program

The analyst will facilitate a discussion on how to implement the metrics program across the organization.

3.7

Sample of activity 3.7 'Maintain the service metrics'. Track and maintain the metrics program

Set up a mechanism to ensure the success of the metrics program by assessing process adherence and process validity.

Insight breakdown

Insight 1

Service metrics are critical to ensuring alignment of IT service performance and business service value achievement.

Insight 2

Service metrics reinforce positive business and end-user relationships by providing user-centric information that drives responsiveness and consistent service improvement.

Insight 3

Poorly designed metrics drive unintended and unproductive behaviors that have negative impacts on IT and produce negative service outcomes.

Summary of accomplishment

Knowledge Gained

  • Follow a methodology to identify metrics that are derived from business objectives.
  • Understand the proper presentation format based on stakeholder needs for information.
  • Establish a process to ensure the metrics provided will continue to provide value and aid decision making.

Processes Optimized

  • Metrics presentation to business stakeholders
  • Metrics maintenance and tracking

Deliverables Completed

  • Metrics Development Workbook
  • Metrics Presentation Format Selection Guide
  • Metrics Tracking Tool

Research contributors and experts

Name Organization
Joe Evers Joe Evers Consulting
Glen Notman Associate Partner, Citihub
David Parker Client Program Manager, eHealth Ontario
Marianne Doran Collins CIO, The CIO-Suite, LLC
Chris Kalbfleisch Manager, Service Management, eHealth Ontario
Joshua Klingenberg BHP Billiton Canada Inc.

Related Info-Tech research

Stock image of a menu. Design & Build a User-Facing Service Catalog
The user-facing service catalog is the go-to place for IT service-related information.
Stock image of a laptop keyboard. Unleash the True Value of IT by Transforming Into a Service Provider
Earn your seat at the table and influence business strategy by becoming an IT service provider.

Bibliography

Pollock, Bill. “Service Benchmarking and Measurement: Using Metrics to Drive Customer Satisfaction and Profits.” Aberdeen Group. June 2009. http://722consulting.com/ServiceBenchmarkingandMeasurement.pdf

PwC. “Mega-Trends and Implications.” RMI Discussion. LinkedIn SlideShare. September 2015. http://www.slideshare.net/AnandRaoPwC/mega-trends-and-implications-to-retirement

PwC. “Healthcare’s new entrants: Who will be the industry’s Amazon.com?” Health Research Institute. April 2014. https://www.pwc.com/us/en/health-industries/healthcare-new-entrants/assets/pwc-hri-new-entrant-chart-pack-v3.pdf

PwC. “Northern Lights: Where are we now?” PwC Blogs. 2012. http://pwc.blogs.com/files/12.09.06---northern-lights-2--summary.pdf

PwC. “PwC’s key performance indicators

About Info-Tech

Info-Tech Research Group is the world’s fastest-growing information technology research and advisory company, proudly serving over 30,000 IT professionals.

We produce unbiased and highly relevant research to help CIOs and IT leaders make strategic, timely, and well-informed decisions. We partner closely with IT teams to provide everything they need, from actionable tools to analyst guidance, ensuring they deliver measurable results for their organizations.

Member Rating

10.0/10
Overall Impact

$1,673
Average $ Saved

57
Average Days Saved

After each Info-Tech experience, we ask our members to quantify the real-time savings, monetary impact, and project improvements our research helped them achieve.

Read what our members are saying

What Is a Blueprint?

A blueprint is designed to be a roadmap, containing a methodology and the tools and templates you need to solve your IT problems.

Each blueprint can be accompanied by a Guided Implementation that provides you access to our world-class analysts to help you get through the project.

Need Extra Help?
Speak With An Analyst

Get the help you need in this 3-phase advisory process. You'll receive 6 touchpoints with our researchers, all included in your membership.

Guided Implementation #1 - Design the metrics
  • Call #1 - Design metrics based on business needs
  • Call #2 - Validate the metrics

Guided Implementation #2 - Design reports and dashboards
  • Call #1 - Select presentation format
  • Call #2 - Review metrics presentation design

Guided Implementation #3 - Implement, track, and maintain metrics
  • Call #1 - Select and implement pilot metrics
  • Call #2 - Determine rollout process and establish maintenance/tracking mechanism

Authors

Valence Howden

Paul Brown

Kimberly Jiang

Contributors

  • Joe Evers, Joe Evers Consulting
  • Glen Notman, Associate Partner, Citihub
  • David Parker, Client Program Manager, eHealth Ontario
  • Marianne Doran Collins, CIO, The CIO-Suite, LLC
  • Chris Kalbfleisch, Manager, Service Management, eHealth Ontario
  • Joshua Klingenberg, BHP Billiton Canada Inc.
Visit our COVID-19 Resource Center and our Cost Management Center
Over 100 analysts waiting to take your call right now: 1-519-432-3550 x2019