IT transformation is hard. You replace slow and error-prone processes in your business with more efficient technology to create more scalable and automated digital workflows. Yet you’re still not sure if you’re “doing” it right. That is, how do you know if your IT transformation is successful?
Most companies rely on financial and operational efficiency metrics to determine if their IT transformation is successful. The problem is that these metrics aren’t always effective in measuring transformational success. Even McKinsey found that less than 15% of companies using financial KPIs can accurately quantify the ROI on their IT transformation investments. Mostly because of the long budget cycle and the lagging response of technology on financial performance.
So, how can your company determine success with your IT transformation initiatives? Let’s look at the top 10 IT transformation metrics that are better suited to tell you whether you’re succeeding with your efforts.
What is IT Transformation?
But before we dive into the numbers, let’s take a quick step back to review what IT transformation is. IT transformation is the process of overhauling the complete IT system of an organisation or company to improve overall efficiency and delivery in a digital economy. It completely modernises a company’s infrastructure, hardware, software, IT service management, data storage frameworks, and IT accessibility.
The Orbis Partnership used an IT transformation initiative several years ago to combine three separate IT departments into one while still serving the needs of each customer base. It implemented AI and automation to clear process backlogs and used chatbots to support HR processes and simple FAQs online.
The British Red Cross worked with WeAreFriday to upgrade their legacy website for the Manchester One Love charity concert in just three days. They built, tested, scaled and deployed the new mission-critical donation website and platform in less than three days. The new platform was mobile-optimised, integrated with Apple Pay, and was built to scale rapidly on Amazon Web Services, something their previous legacy platform couldn’t do. The new platform was so resilient that it hosted 33,000 concurrent users at one point, making up to 823 donations a minute, and raised over £475,000 in just three hours.
The difference between IT and digital transformations
Just a quick note on the difference between transformations:
- An IT transformation is the foundation on which a digital transformation (DX) is built, but it can be done separately from a DX initiative.
- A DX initiative is more about rethinking processes and business models to become more customer-centric.
But a DX initiative cannot happen without an IT transformation.
IT Transformation Metrics to Measure Success
When it comes to measuring success, most IT transformation initiatives use a fairly simple pass/fail approach. Positive numbers mean that all is well with the measured IT operations and services. Negative numbers warn that the initiative may be headed in the wrong direction or is actively damaging your business goals.
But an IT transformation isn’t a pass/fail type of initiative, so let’s look at the top 10 metrics you can use to measure success. You’ll notice that we’ve grouped some of the metrics because there might be several ways to track a metric’s concept. You can choose the ones that work best for your company and business.
1. Active Usage
A successful transformation requires sustainable adoption of the new technology, so measuring active usage among registered users and systems is helpful. You’ll want to measure:
- Daily active users
- Ratio of new to repeat users
- Conversion and abandonment rates
For example, if you see that more than 85% of users are regularly active on the new systems, you can assume that things are going well. Or, in the case of the British Red Cross and their new donation platform, if abandonment rates were high, that meant the new platform wasn’t working properly.
2. User Engagement
Engagement can be a bit of a nebulous word to think about in terms of metrics, but it’s useful when measuring IT transformation success. For example, if bounce and exit rates for a web-based application are poor, that could indicate an engagement issue. It’s not that users don’t want to use the new technology; it could be there’s a tech issue that’s preventing them from completing the task or process it’s intended for.
Net promoter scores (NPS) and customer satisfaction rates can help indicate whether the new technology is meaningful to users and help them shift their existing work patterns. Traffic source metrics outline where engagement comes from, whether from the intended sources or new ones. For example, perhaps a content management system was deployed to help marketing teams but based on the traffic source numbers, you may find that your sales and customer service teams are using it more.
3. User Adoption
Adoption rates tell you how well employees and systems are using your IT transformation tools and technology. Poor initial adoption rates could happen because the new tool is complex to use for new users or because it wasn’t really necessary in the first place. You may think a replaced tool wasn’t being used simply because it wasn’t modern enough for your business. The adoption rates might tell you otherwise.
4. Customer Experience
If your initiative touches on customer-facing technologies or systems, you’ll want to evaluate its customer perception. Metrics such as customer satisfaction rates, customer effort scores, and customer loyalty index rates will help you understand the effectiveness of your changes. They’re especially useful if you’re doing a multi-phase IT transformation and will be rolling out new technologies in succession to your customers.
5. Service Desk Calls
Replacing old technology or adding a new system almost always leads to increased IT service desk calls. Some calls may be due to the complexity of the new application or because it’s not functioning properly (e.g., a data store that’s not connecting to related systems or a bottleneck somewhere causing significant lag for users.)
Metrics to track here include the number of tickets created and resolved, call wait times, ticket growth over time, and top IT services or apps with tickets.
6. Availability and Reliability
Transforming operational processes and workflows that rely on legacy systems or manual processes can lead to reliability issues. For example, migrating a legacy component could cause connectivity issues further down the line or break workflows because of unexpected bottlenecks.
Measuring your new IT system performance before, during, and after the transition will help you discover these insights. Common metrics to track include: system uptime, mean time to failure (MTTF,) mean time to resolve (MTTR,) and downtime duration.
7. Risk Factors
One of the biggest reasons for starting an IT transformation initiative is to reduce the risk associated with your technology. As more of your business goes digital, you’ll need to continually evaluate the risks surrounding your existing technology and any new ones you bring in, especially regarding cybersecurity and data privacy.
Metrics to track here would include data transfer volumes before and after the transformation, the number of systems in your network with known vulnerabilities, the number of privilege levels your systems have and how many you truly need, and the frequency of planned and unplanned outages.
8. Cost and Revenue Indicators
Upper management will always want to know how much the IT transformation costs and how it’ll affect the bottom line, so tracking these metrics is important too. However, it’s not just enough to track the final costs of the initiative. Combining cost metrics with adoption, usage, and customer experience metrics will give you a better understanding of how effective your changes have been.
For SaaS and other subscription-based service companies, you’ll want to track metrics like:
- Monthly recurring revenue (MRR) before and after the change
- Number of support tickets created
- Churn rates
- Customer lifetime value (CLV)
- Contraction MRR
9. Time to Deliver Defined Business Goals
Another metric to measure is the time to deliver on a defined business goal. Since the point of transforming your technology is to improve on workflows, costs, and performance, you’ll want to know how those changes are directly impacting your business goals.
This might be a challenging metric or approach to “sell” to the IT team since they’re not always aware of how their work impacts the company’s bottom line. Aligning the IT transformation with business goals will help publicise its effect and ensure everyone is working towards the same goals.
10. Service Level Compliance
Service level agreements (SLAs) oblige vendors to deliver their services or technology according to documented levels defined by specific metrics (usually included in a vendor agreement or contract.) Failure to comply with these SLAs has financial consequences for vendors but also for you, as the customer. Your end users, employees, and connected systems are impacted by any downtime, performance issues, or failure by the vendor to fix issues. So it’s important to identify the SLA metrics appropriate for your business and IT transformation.
Some example SLA metrics include:
- Time to first response: How fast the vendor responds to a support request.
- Average handling time: How long it takes for the vendor to resolve a request.
- SLA compliance ratio: The ratio between the number of incidents resolved in compliance with SLA and the total number of incidents
Measuring Success Starts with the Right Metrics
When you know how your company reacts to the initiative, you can pivot and course-correct as needed to ensure success. Measuring the right metrics will help you find the right inflexion point for your IT transformation and guide you to success. You’ll know if your changes are well aligned with your business goals, how your workflows and processes are affected, and how your employees are dealing with all the changes.
Metrics and KPIs are useful tools, but only if you’re measuring the right ones. Use this list of the top 10 metrics before, during, and after your IT transformation initiative to evaluate how it’s doing.
