PRAGMATIC Security Metric of the Quarter #3

PRAGMATIC Security Metric of the Third Quarter

These are the example information security metrics we have discussed and scored over the past three months, ranked in descending order of their PRAGMATIC scores: 

Example metric	P	R	A	G	M	A	T	I	C	Score
Metametrics	96	91	99	92	88	94	89	79	95	91%
Access alert message rate	87	88	94	93	93	94	97	89	79	90%
Asset management maturity	90	95	70	80	90	85	90	85	90	86%
Compliance maturity	90	95	70	80	90	85	90	85	90	86%
Physical security maturity	90	95	70	80	90	85	90	85	90	86%
Thud factor	82	80	60	60	70	45	85	86	84	72%
Business continuity spend	75	92	20	82	95	70	70	70	70	72%
Benford's law	84	30	53	95	11	98	62	98	23	62%
Controls coverage	87	89	65	40	74	35	46	40	30	56%
Homogeneity	67	70	40	59	67	50	33	65	45	55%
Access control matrix status	70	50	60	60	88	25	40	20	40	50%
Unaccounted software licenses	1	1	90	84	1	70	50	81	30	45%
Unauthorized/invalid access count	61	78	33	16	33	0	44	35	33	37%

On that basis, we are happy to announce that the Information Security Metric of the Quarter is <cue drum roll> metametrics meaning a systematic measure of the quality (utility, value, fitness for purpose etc.) of the organization's information security metrics.  

Clearly we have in mind here the PRAGMATIC approach, but in fact other/variant approaches are possible, so long as there is a sensible, rational way of assessing the metrics that makes sense to management.  The key point of the metric, and the reason it scores so highly, is that measuring the quality of the organization's security metrics is an important step enabling management to improve the suite of metrics in a systematic manner, rather than the much more common ad hoc approach.  Better information security metrics, in turn, allows management to get a firm grip on the organization's information security arrangements, bring them under control, and improve them systematically too.  There is in fact a positive feedback loop at play, since better, more reliable and suitable information security arrangements generate better, more reliable and suitable data concerning information security risks and controls, in other words better metrics. 

That said, we fully accept our own obvious bias in this matter.  Having invented and written about the PRAGMATIC approach, we inevitably see metametrics through rose-tinted glasses.  Things may not be quite so rosy from your perspective, and that's fair enough.  But when you have a moment to yourself, take another look at the 13 metrics on the summary table above, plus those covered in the previous 2 quarters (browse back through the blog, or visit the Security Metric of the Quarter #1 and Security Metric of the Quarter #2), and draw your own conclusions.  You probably disagree with us on the scoring of some of the metrics (even in the hypothetical context of an imaginary company).  But, overall, do you accept that this is a reasonably straightforward, sensible way to consider, compare and contrast metrics?  Would you agree that, on the whole, the metrics that score well on the PRAGMATIC scale are better than those that score badly?  Is this discussion about the pros and cons of security metrics, using metametrics, something that you might use back at the ranch?

Bottom line: if we have persuaded you that the PRAGMATIC approach has merit, perhaps even that it might be a valuable addition to your arsenal of security management techniques, read the book for the full nine yards.  This blog is just a taster of what's to come.

Gary & Krag