25 May 2015

Low = 1, Medium = 2, High = 97.1

Naïve risk analysis methods typically involve estimating the threats, vulnerabilities and impacts, categorizing them as low, medium and high and then converting these categories into numbers such as 1, 2 and 3 before performing simple arithmetic on them e.g. risk = threat x vulnerability x impact.

This approach, while commonplace, is technically invalid, muddling up quite different types of numbers:
  • Most of the time, numeric values such as 1, 2 and 3 are cardinal numbers indicating counts of the instances of something. The second value (2) indicates twice the amount indicated by the first (1), while the third value (3) indicates three times the first amount. Standard arithmetic is applicable here.
  • Alternatively, 1, 2 and 3 can indicate positions within a defined set of values - such as 1st, 2nd and 3rd place in a running race. These ordinal values tell us nothing about how fast the winner was going, nor how much faster she was than the runners-up: the winner might have led by a lap, or it could have been a photo-finish. It would be wrong to claim that the 3rd placed entrant was “three times as slow as the 1stunless you had additional information about their speeds, measured using cardinal values and units of measure: by themselves, their podium positions don’t tell you this. Some would say that being 1st is all that matters anyway: the rest are all losers. Standard arithmetic doesn't apply to ordinals such as threat values of 1, 2 or 3.
  • Alternatively, 1, 2 and 3 might simply have been the numbers pinned on the runners’ shorts by the race organizers. It is entirely possible that runner number 3 finished first, while runners 1 and 2 crossed the line together. The fourth entrant might have hurt her knee and dropped out of the race before the start, leaving the fourth runner as number 5! These are nominals, labels that just happen to be digits or strings of digits. Phone numbers and post codes are examples. Again, it makes no sense to multiply or subtract phone numbers or post codes. They don’t indicate quantities like cardinal values do. If you treat a phone number as if it were a cardinal value and divide it by 7, all you achieved was a bit of mental exercise: the result is pointless. If you ring the number 7 times, you still won’t get connected. Standard arithmetic makes no sense at all with nominals.
When we convert ordinal values such as low, medium and high, or green, amber and red, risks into numbers, they remain ordinal values, not cardinals – hence standard arithmetic is inappropriate. If you convert back from ordinal numbers to words, does it make any sense to try to multiply something by "medium", or add "two reds"? “Two green risks” (two 1’s) are not necessarily equivalent to “one amber risk” (a 2). In fact, it could be argued that the risk scale is non-linear, hence “extreme” risks are materially more worrisome than most mid-range risks, which are of not much more concern than low risks. Luckily for us, extremes tend to be quite rare! As ordinals, these risk numbers tell us only about the relative positions of the risks in the set of values, not how close or distant they are – but to be fair that is usually sufficient for prioritization and focus. Personally, a green-amber-red spectrum tells me all I need to know, with sufficient precision to make meaningful management decisions in relation to treating the risks.

Financial risk analysis methods (such as SLE and ALE, or DCF) attempt to predict and quantify both the probabilities and outcomes as cardinal values, hence standard arithmetic applies … but don’t forget that prediction is difficult, especially about the future (said Neils Bohr, shortly before losing his shirt on the football pools). If you honestly believe your hacking risk is precisely 4.83 times as serious as your malware risk, you are sadly deluded, placing undue reliance on the predicted numbers.

No comments:

Post a Comment

Have your say!