Naïve risk analysis methods typically involve estimating the threats, vulnerabilities and impacts, categorizing them as low, medium and high and then converting these categories into numbers such as 1, 2 and 3 before performing simple arithmetic on them e.g. risk = threat x vulnerability x impact.
This approach, while commonplace, is technically invalid, muddling up quite different types of numbers:
- Most of the time, numeric values such as 1, 2 and 3 are cardinal numbers indicating counts of the instances of something. The second value (2) indicates twice the amount indicated by the first (1), while the third value (3) indicates three times the first amount. Standard arithmetic is applicable here.
Alternatively, 1, 2 and 3 can indicate positions
within a defined set of values - such as 1st, 2nd and 3rd
place in a running race. These ordinal values tell us nothing
about how fast the winner was going, nor how much faster she was than the
runners-up: the winner might have led by a lap, or it could have been a
photo-finish. It would be wrong to claim that the 3rd placed
entrant was “three times as slow as the 1st” unless you had
additional information about their speeds, measured using cardinal values and
units of measure: by themselves, their podium positions don’t tell you this. Some would say that being 1st is all that matters anyway: the rest are all losers. Standard arithmetic doesn't apply to ordinals such as threat values of 1, 2 or 3.- Alternatively, 1, 2 and 3 might simply have been the numbers pinned on the runners’ shorts by the race organizers. It is entirely possible that runner number 3 finished first, while runners 1 and 2 crossed the line together. The fourth entrant might have hurt her knee and dropped out of the race before the start, leaving the fourth runner as number 5! These are nominals, labels that just happen to be digits or strings of digits. Phone numbers and post codes are examples. Again, it makes no sense to multiply or subtract phone numbers or post codes. They don’t indicate quantities like cardinal values do. If you treat a phone number as if it were a cardinal value and divide it by 7, all you achieved was a bit of mental exercise: the result is pointless. If you ring the number 7 times, you still won’t get connected. Standard arithmetic makes no sense at all with nominals.
When we convert
ordinal values such as low, medium and high, or green, amber and red, risks
into numbers, they remain ordinal values, not cardinals – hence
standard arithmetic is inappropriate. If you convert back from ordinal numbers to
words, does it make any sense to try to multiply something by
"medium", or add "two reds"? “Two green risks”
(two 1’s) are not necessarily equivalent to “one amber risk” (a 2). In
fact, it could be argued that the risk scale is non-linear, hence “extreme”
risks are materially more worrisome than most mid-range risks, which are
of not much more concern than low risks. Luckily for us, extremes tend to
be quite rare! As ordinals, these risk numbers tell us only about the
relative positions of the risks in the set of values, not how close or distant
they are – but to be fair that is usually sufficient for prioritization and
focus. Personally, a green-amber-red spectrum tells me all I
need to know, with sufficient precision to make meaningful management decisions in
relation to treating the risks.
Financial risk analysis methods (such as SLE and ALE, or
DCF) attempt to predict and quantify both the probabilities and outcomes as
cardinal values, hence standard arithmetic applies … but don’t forget that
prediction is difficult, especially about the future (said Neils Bohr, shortly before
losing his shirt on the football pools). If you honestly believe your
hacking risk is precisely 4.83 times as serious as your malware risk, you are sadly deluded, placing undue reliance on the predicted numbers.
