Metrics can be incredibly valuable, but also incredibly easy to get wrong. Of the plethora of available IT metrics, risk metrics are probably the most difficult to get right. This is mainly due to the fact that risk is tricky to define and therefore tough to measure.
You Can’t Manage What You Can’t Measure
Defining the word ‘risk’ itself is actually quite simple. In a nutshell, risk is the probability/likelihood of a negative event occurring multiplied by the magnitude of the impact/loss that could happen as a result of said event. Sounds easy enough, right? Not so fast…
Risk must be measured in a way that is meaningful to the organization. A good risk metric demands that you aggregate numerous measurements in order to create a piece of intelligence that’s actually useful to both IT and the business. As I mentioned in an earlier blog post on root cause analysis, accurate measurement means asking the right questions:
- What problem is the risk metric addressing?
- Which decision point(s) does the risk metric support?
- What context should the metric take into account?
Issues arise when these questions are not rigorously applied to each and every risk metric under consideration for deployment.
The inclusion or exclusion of certain loss events (say, an asteroid wiping out your data center) are going to determine whether or not the risk metric itself has any meaning. Take the example of using a very basic risk formula to help flesh out which precautions should be taken in the IT disaster recovery plan:
% chance of loss event x $ loss magnitude
Sure, a meteor strike would probably have the highest dollar cost in terms of magnitude, but the likelihood of such an event is about as close to zero as you can get without falling in. I’m oversimplifying for dramatic effect here, but I hope you can see that such a risk metric utterly fails to take into account the three questions about problem, decision, and context.
Choosing Metrics That Matter
Here are a few takeaways to think about as you plan risk metrics for your organization.
1. Always consider your audience. Metrics for risk are high-level enough that they are probably being communicated to executives and others of that ilk. At lower levels, it’s all about IT managers and domain-specific professionals or support staff. Meet with these stakeholders on a regular basis to understand which metrics are the most important to them. For example:
2. Determine the suitability of each risk metric. This is where the three questions come into play. Once you’ve settled on the audience you’re delivering metrics reporting to, apply the questions accordingly. Let’s use security performance trends as an example:
- Problem – how to determine effectiveness of the security program
- Decision – whether or not to continue with certain program elements
- Context – executives believe it’s just as important to measure failures
3. Tie risk metrics back to business objectives. Business objectives could include maximizing IT investments, using risk management to adapt to changing regulations, or a host of other business concerns. Whatever the objectives are, your risk metrics are going to be rendered useless if they’re not aligned with corporate strategic initiatives.
Obviously there’s a lot more to risk metrics than what I’ve talked about here. Please refer to the links below for more in-depth guidance.
Related Info-Tech Resources