Metric of the Week #50: policy clarity

Information Security Metric of the Week #50: proportion of information security policies that are clear

This week's worked example concerns a metric that looks, at first glance, to be far too subjective to be of any value ... but read on.

Clarity is a critical requirement for information security and indeed other kinds of policies.  Policies that are garbled, convoluted, rambling and full of jargon are less likely to be read and understood by the people that ought to be compliant.  As a corollary, well-written, succinct policies facilitate reading and understanding, while also making them 'motivational' in style encourages compliance.

It's obvious, isn't it?  So how come we still occasionally see policies written in the worst kind of legalese, stuffed with obscure/archaic language in an embarrassingly amateurish attempt, presumably, to appear "official"?

The suggestion for this metric involves regularly surveying employees' opinions regarding the clarity of the security policies, using a questionnaire based on a Likert scale developed and administered in person by a market research firm.   

Applying the PRAGMATIC method, ACME managers expressed some interest in this metric, but were concerned on a number of fronts:

P	R	A	G	M	A	T	I	C	Score
75	70	68	41	96	50	56	90	34	64%

There are at least three advantages to this metric, the first one more obvious than the others:

1. Aside from the overall clarity/readability level, the metric should identify policies that are perceived as better or worse than average, providing shining examples and opportunities for improvement, respectively.  This is useful new information for the authors of policies, making this a beneficial operational security metric. 
2. Asking employees about the policies will inevitably find some unable or unwilling to state an opinion because they cannot recall reading the policies.  The proportion of 'no response' returns is therefore a rough measure - an indicator - of the extent to which policies are actually being read.
3. Asking employees about policies also prompts them, and hopefully their colleagues, to (re)read the policies in order to give an opinion ... which is itself a useful awareness outcome.

On the downside, however, the metric would be Costly and slow to collect if a market research company was engaged.  It could be run more cheaply by employees such as Information Security, although since they are generally responsible for the policies, they may be tempted to influence or manipulate the metric to make them appear clearer than they really are - or at least they would find it difficult to prove that they were completely unbiased.  Increasing the Cost-effectiveness rating in this way would therefore depress the Independence factor and, perhaps, the Genuinness and Accuracy factors which are already low due to this being such a subjective matter.

There is a further advantage to having members of Information Security conduct the surveys in person: it would give them a legitimate reason to leave the sanctuary of the security office, get out among their colleagues and pick up on things that are really going on in the business.  Their colleagues, at the same time, would get to meet and chat with Information Security people in an informal, non-confrontational situation, and could perhaps raise other queries or concerns.  This would be particularly valuable if the security team was reclusive and shy, or appeared aloof and distant.

[A variant of this metric involving an automated assessment of clarity was also tabled for consideration by ACME management: it will appear on this blog at some point so you can either wait patiently to find out about it or look it up in Chapter 7 of the book!]