SMotW #79: Employee turn vs account churn

Security Metric of the Week #79: Employee turn versus account churn

This week's metric is typical of the kind of thing that often crops up in security metrics workshops and meetings. Whenever someone invents or discovers a metric like this, they are often enthusiastic about it, and that enthusiasm can be infectious. 

The alliteration in 'employee turn versus account churn' is eye-catching: for some reason buried deep in the human psyche, we find the phrase itself strangely attractive, hence the metric is curiously intriguing. 

We've fallen into a classic trap: the metric sounds 'clever' whereas, in reality, this is a triumph of form over substance. It is far from clear from the cute phrase what the metric is actually measuring, how, why, and for whom. What are 'employee turn' and 'account churn', exactly, and why would we want to compare them? What would that tell us about information security anyway?

In practice, someone at the workshop would probably have asked questions along those lines of the person who proposed the metric, and in turn they would have made a genuine attempt to explain it. In a field as complex as this, it's really not hard for an enthusiastic and influential person to concoct an argument justifying almost any security metric.  Combine that with a team exhausted by discussing dozens of metrics candidates, and it's easy to see why rogue metrics might slip through to the next stage of the process: management review.

By forcing this metric through the PRAGMATIC sausage machine, ACME's managers stripped back the gloss to consider its potential as a means of measuring information security:

P	R	A	G	M	A	T	I	C	Score
30	30	11	36	44	36	62	57	20	36%

Strangely, despite marking the metric down on Predictiveness, Relevance, Actionability, Accuracy, and Cost-effectiveness, they thought it had some Meaning. Perhaps they too were intrigued by the alliterative phrase! Nevertheless, the metric's poor overall score sealed its fate since there were many stronger candidate metrics on the table.

Remember this example whenever someone proposes a 'clever' security metric. Is it truly insightful, or is it simply obtuse and perplexing? By the same token, think twice about your own pet security metrics - and yes, we all have them (ourselves included!). 

Taken in the proper sequence, the Goal-Question-Metric approach forces us to start by figuring out what concerns us and then pose the obvious questions before finally considering possible metrics. Rogue metrics are less likely to crop up and harder to explain and justify. PRAGMATIC filters out any that make it through the earlier screening, despite their being pushed by influential people who are infatuated with their pets. This may seem rather cold and sterile, but think about it: metrics are all about bringing cool rationality, precision and facts to the management of complex processes. There's no room for rogues.