Over several IT Quality Index projects, we have had a great opportunity to discuss the key issues which are facing IT managers on different
levels.
CIOs need to know:
- Is our IT good or not good?
- How to measure and compare technology capabilities against others?
- What is the right metric for our executives?
One very common approach is to use some method of maturity assessment or benchmarking. Let’s make a brief review of both:
Maturity assessments are usually structured per individual processes, where the output is the maturity levels of individual processes. This is a good approach when IT is like a factory, with a high volume of consistent, repetitive activities, unlikely to evolve over time, which doesn’t sound very common in today’s world of constant change.
The key point here is that executives don’t understand what your processes are, and even less what the process maturity score means for them and their teams.
Benchmarking is usually focused on some frequently measured parameter – like average costs per resolved incident, average costs per end user. While this might look reasonable, the fundamental problem is, that those metrics tell us nothing about overall quality, so we have seen “efficiency leaders” who have the very worst management of IT.
Beside of key issues described above, we still didn’t touch an even bigger more significant problem with measurement. How can we measure the energy efficiency of measurement?
Imagine a situation where a CIO spends a huge sum of money for a detail process maturity assessment, but the benefits are doubtful. Do the measurement costs translate into something useful? And useful for what and to who? How do we measure the usefulness of the measurement exercise?
We need to open a new debate about the costs and benefits of the measurements we use. How can we quantify if the results of our measurement are well balanced against costs… The answer is you can’t, it’s not possible unless we compare different methods and start to capture costs/benefits feedback from CIOs.
And this is what we do during an IT Quality Index assessment. After handing over the measurement results and charging the client we ask for feedback. “How would you compare costs paid for IT Quality Index measurement with other assessment methods you used so far?”.
The very typical answer to our question is:
Client CIO: ”ITQI assessment was much less demanding on our resources, less expensive and provided us with a better understanding of where we are and how we compare from a costs/quality perspective. As the output is easier to understand than other assessments we used, we estimate that measurement energy and costs consumption is on 30-40% of what we experienced so far.”
We continue to collect more and more feedback and have built up enough knowledge and experience to draft proposals regarding measurements and the real cost to your business.
Every assessment method should be capable of defining resource requirements for the measurement itself. Every measurement model designer, consultancy firm and benchmarking company should think about it as an energy consumption model. Which kind do you use?
For CIOs – next time, when you consider any assessment model, you should ask your consultancy partner for the energy rating of their method, if they can’t provide it, what does that say?
IT Quality Index is a new method of how to measure and visualize IT quality. It measures 48 quality dimensions, in a very costs efficient way, giving you clearer results that translate to action without the heavy consumption of resources.
The method is documented in the book IT Quality Index, ISBN 9789401802420
Overview of IT Quality Index is here.
BTW: How energy efficient is your IT Governance?