Why it’s time to ditch Probability Impact Graphs.
by Tony Badsey-Ellis, Risk Consultant.
Probability/impact graphs have been used for a long time to assess risk, especially in spreadsheet-based risk registers. They give a misleading impression of risk levels though and hence organisations should be using alternative, and better approaches.
Two key reasons to stop using PIGS now:
- Not all the numbers in the grid are used.
- Risks on the same diagonal give the impression that they are the same, yet can have significant differences in their scores.
What is a PIG?
For many years, risk assessments have used a standard(ish) formula for ‘calculating’ risk level. A two-dimensional matrix is used, labelled impact on one axis, and probability on the other. For each risk, an assessment of each factor is made, and the risk is plotted into the matrix. There is almost always a coloured background on the matrix as well, with a rainbow from red to green sweeping diagonally across. Most of the matrices have this as a symmetrical pattern, with a triangle of reds in the corner for highest impact and probability, and a triangle of greens opposite. They are sometimes referred to as Probability/Impact Graphs (or PIGs for short).
The trouble with PIGs.
A risk level is calculated for each risk by multiplying the scores for impact and probability. The trouble is, this is really crude. Most of the matrices are 5×5 or 6×6, giving just 25 or 36 possible positions for the risks. The colours are usually symmetric, which in effect states that probability and impact are equally important. This also reduces the actual possible range of positions to 15 or 21.
Once the scores are calculated they are then put back into the risk register. Anyone reading this without a knowledge of the matrix process might therefore assume that there is a range of possible scores from 1 (lowest) to 25 or 36 (or possibly lower, if there aren’t any risks at the highest level of both impact and probability). They would be wrong.
The scores are calculated by multiplying two whole numbers together. It is therefore impossible to get a score that is a prime number greater than 5. There will also be some non-prime scores that cannot be reached because of the short axes on the matrix. On a 6×6 you cannot score a risk as 14, 21, 22, 26, 27, 28, 32, 33, 34, or 35, all of which are non-prime. Together with the prime numbers, on this size matrix it is impossible to use 50% of the values between 1 and 36.
Is the answer to get bigger PIGs (a larger matrix)?
In short, no. The larger the matrix, the worse this problem becomes. By the time the matrix reaches 10×10, only 42% of the possible values are used, and it just gets worse the larger the matrix becomes.
The distribution of the numbers is an even bigger problem. Fourteen of the cells on the 5×5 matrix score 8 or less; eight cells score between 9 and 17, and the remaining three cells score 18 or higher. So, the lowest one-third of the possible scores occupy 56% of the matrix, and the highest third just 12%. And, as with the number of possible scores, the distribution gets worse with larger matrices. For a 10×10 matrix the lower, middle, and upper thirds occupy 63%, 27%, and 10% of the matrix respectively.
Any other issues with PIGs?
OK, so that’s the techie statistical stuff. But if we’re happy to accept these issues, and just focus on the scoring, is there still a problem? Well, yes, and it’s because of how the PIGs work. Think about a 6×6 matrix with even bands striped across it.
Since the risk scores are calculated by multiplying probability and impact, those two factors are weighted the same. So, a risk that has values of (6, 1) – plotted in the bottom-right square, will score 6. If the probability is reduced by one and the impact increased by one [the solid blue arrow] then the risk should be the same (due to the equal weighting). But our new risk plots at (5, 2), so scores a 10. It now appears to be 40% higher than before. It doesn’t matter if we switch probability and impact – they have the same weighting, and so the same will happen with the scores.
If we repeat the process of moving the risk diagonally [the dotted blue line] it proceeds to (4,3), scoring 12 – twice the score of the original cell. The next move keeps the risk score the same, and then it drops back to 10 and then 6. Each time we perform the same process, but the final scores are affected differently. This is no way to do risk management!
PIGs are often the result of using a spreadsheet to catalogue risks. This is a poor way to understand risks, as it tends to be created using a structureless process that misses risks. Once in a spreadsheet, quickly adding scores for probability and impact is easy, and they can then be plotted onto a PIG, giving the illusion of having used a mathematical approach to risk management.
Using attack trees for a more rigorous approach to risk assessment
Instead of lulling yourself – and your organisation – into a false sense of security (!) using PIGs and spreadsheets, why not look at a more comprehensive and rigorous approach to risk management? RiskTree is a structured process that uses attack trees to model your risks and quickly generate a detailed analysis that can be easily and efficiently updated as your system evolves.
The National Cyber Security Centre (NCSC) recently published new risk management guidance, highlighting the use of attack trees as a powerful tool for understanding and addressing cyber security risks – you can read more about it in our blog.
Interested in trying RiskTree for free? Please register here.
If you have any questions about risk, security architecture or security monitoring – you can contact us here.