SMotW #37: unaccounted software licenses

Security Metric of the Week #37: proportion of software licenses purchased but not accounted for in the repository

We are not entirely sure of the origin or purpose of this metric, but it's typical of the those that pop randomly out of the woodwork every so often for no obvious reason, sometimes taking on a curious aura of respectability depending on who raised or proposed them.  

Unfortunately, as it stands, we lack any context or explanation for the metric.  We don't have access to whoever proposed it, we can't find their reasoning or justification, and hence we find it hard to fathom their thinking processes that presumably led them to propose it.

Perhaps someone had been checking, validating or auditing software licenses and  used something along these lines as a measure in their report.  Maybe it was suggested by a colleague at an information security meeting or online forum, or proposed by a naive but well-meaning manager in such a way that it simply had to be considered.  Who knows, perhaps it came up in idle conversation, mystically appeared out of the mist in a dream, turned up as a worked example in a security metrics book, or featured in some metrics catalog or database.  

It may well have been someone's pet metric, something they invented, discovered or borrowed one day for a specific purpose, found useful in that context, and so presumed their success means it must therefore be a brilliant security metric for everyone, in other, unspecified contexts.*  

To be frank, we not terribly bothered where it came from or why it appeared on our shortlist.  We do care about its utility and value as a security metric for ACME Enterprises Inc, in relation to the plethora of others under consideration.

Maybe for some it really is a wonderful metric ... but evidently not for ACME.  The PRAGMATIC score says it all:

P	R	A	G	M	A	T	I	C	Score
1	1	90	84	1	70	50	81	30	45%

It scores abysmally on Relevance (to ACME's information security), on its ability to Predict or be used to direct ACME's information security status, and on its Meaning to ACME's information security people and managers.  On the other hand, it is highly Actionable in the sense that a low score self-evidently  implies the need to account for more of the purchased software licenses.  It's also pretty Genuine and would be hard to falsify unless someone had the motivation, skill and time to fabricate a stack of 'evidence' from which the numbers could be reconstructed.  ACME's people have better things to do.

OK so it's not ideal for information security but maybe it would have more value to, say, Finance or IT?  Perhaps they too could be persuaded to PRAGMATIC rate the metric and compare it to those they are using or considering ... no promises, mind you.

Anyway, its poor score clearly takes it out of contention as an information security metric for ACME, and right now we have a date with a mince pie and a small glass of vintage port ...

Merry Christmas readers.

* Note that we are not immune from this kind of generalization and a bias towards the metrics that we find valuable.   The metrics in the book, including the 'security metrics of the week' on this blog, come from a variety of sources.  Some were metrics that we have used in anger ourselves, before, including a few of our own pet metrics of course.  Some have been suggested, recommended even, by various other security metrics authors.  Some made an appearance in security surveys, management reports, blogs, discussion groups and standards such as ISO/IEC 27004.  Some we invented on-the-fly while writing the book, deliberately trying to illustrate and demonstrate the power of the PRAGMATIC approach in helping to differentiate the good from the bad and the ugly.  

Please remember, above all else, that whatever we or others may say or imply, we are NOT telling you what security metrics to use in your situation.  We are not clairvoyants.  We have ABSOLUTELY NO IDEA what your specific security information needs might be, except in the most general hand-waving sense of being infosec greybeards ourselves.  Much as we would love to just give you "the best security metrics" or a set of "recommended" or "valuable" or "worthwhile" metrics, we honestly can't do that.

What we are offering is a
straightforward method for you to
find your own security metrics.

In the unlikely event that you are short of inspiration, the book includes a stack of advice on where to find candidate security metrics - places to go looking - and hints on how to invent new ones either from scratch or by modifying and customizing or adapting existing or proposed metrics.  The PRAGMATIC method is a great way to sift through a giant haystack of candidate security metrics to find the very needles you've been hunting for.