SMotW #6: Policy coverage

Security Metric of the Week #6: Information security policy coverage

Corporate information security policies don't normally exist in splendid isolation but to some extent build upon internal and external sources such as:

• Identified information security risks (threats, vulnerabilities and potential impacts) or issues of concern to the organization; 
• Other policy statements and/or other requirements mandated by management;
• Security-relevant compliance obligations imposed by applicable laws, regulations, contracts, agreements, moral codes etc.;
• Good practice security advice from public information security standards, models and frameworks such as ISO27k and the NIST SP800-series, plus the vendors of IT system and software, consultants, textbooks, industry advisories etc., plus of course the advice of competent and experienced employees (e.g. IT audit, risk management and information security professionals).

This week's example metric therefore seeks to measure coverage of the organization's security policies - the extent to which they take account of applicable issues and requirements.

There are several ways in which this metric might be measured in practice, depending on available resources and on the particular aspects that are of most value to management for policy-related decisions.  A common approach is to draw up a two-dimensional matrix (i.e. a table!) listing out the requirements from each source (normally as columns), and identifying the corporate policies, standards, procedures, contracts, agreements etc. that cover them (normally as rows).  A simple traffic-light color scheme in the body of the matrix will suffice to identify requirements that are fully covered by the corporate policies etc. (green), partially covered (amber), not covered (red) or not applicable (clear).  

While the resultant "heat map" is perfectly adequate as a reporting and management tool, some might prefer to count and report the number or proportion of reds, ambers and greens, get more sophisticated using percentages and weightings (since some requirements are trivial whereas others may be crucial), or perhaps identify the number of months that have passed since each requirement was identified and has not yet been fully addressed.  These elaborations will increase the Cost of the metric and may affect its Meaningfulness (the numbers will have to be explained, but the additional information may be valued by management).

P	R	A	G	M	A	T	I	C	Score
75	82	92	78	80	70	73	60	81	77%

We gave this metric a healthy score of 77% on the PRAGMATIC scale.  It is certainly Relevant to security and Actionable.  For example, if an additional security requirement emerges as a result of, say, risk analysis or a new law, it will initially be identified on the matrix as red blobs against the policies, procedures etc. that need to be updated.  These will gradually turn green as the updates are completed.  

Do you measure policy coverage?  If so, how do you do it?  How useful do you find the metric?  We'd love to find out from you how it works out in practice.