SMotW #73: Psychometrics

Security Metric of the Week #73: Psychometrics

We've cheated a bit with this week's example metric: 'psychometrics' such as OCAI, MBTI and MSCEIT are actually an entire class of metrics rather than one in particular. The discussion that follows concerns psychometrics in a general sense.

Many of us will have been invited to take psychometric tests during the job application and interview process. Psychometric testing is based on the science of psychology. The tests provide an additional source of information about candidates' psychological makeup - personalities, attitudes, aptitudes and so on - the 'soft stuff' that is important for almost all positions but which is hard to gauge from résumés or (unskilled/untrained) interviewers. 

Given that a substantial part of information security revolves around human behaviors and attitudes (such as ethics or compliance with policies), ACME's CISO wondered if psychometrics might have potential as security metrics. Using the PRAGMATIC method, ACME's managers gave a straight answer:

P	R	A	G	M	A	T	I	C	Score
40	24	0	79	15	55	10	42	5	30%

Maybe the managers were feeling distinctly cynical and jaundiced when they went through the process: it's unusual for them to assign a zero rating, in this case for Actionability. Their stated rationale was that a person's psychological profile is an inherent and immutable part of their personality, not amenable to being adjusted or managed in an active sense.  They were also concerned at escalating Costs if psychometric testing was extended beyond the recruitment process for information security purposes.

The CISO was tempted to counter that technically it is possible to influence someone's personality to some extent, or at least to influence our natural/preferred behavior patterns for example through training, supervision and guidance. Furthermore, that is not the only way in which psychometrics might be acted upon. They are already commonly used to support hiring decisions, but there may be other opportunities to use psychometrics, for instance for annual appraisals and/or when considering whether to promote or transfer employees. Conceivably, some personality types or characteristics might be ideally suited to high-trust security-relevant roles, while others might be indicators of trouble ahead, but without further research, the CISO would definitely be stepping out on a limb if anyone asked for specifics.

However, rather than thrash out that point and various other issues with what appeared to be a tired and grumpy ACME management, the CISO decided discretion was the better part of valor. The low ratings and pathetic overall score meant this metric was very unlikely to fly, especially given that there were many other higher-scoring metrics already on the table. That's not to say psychometrics would never be a worthwhile security metric, nor that they are necessarily a poor choice for your organization, rather that ACME was simply not ready for them.  Yet.

[Whereas mostly we talk about the PRAGMATIC method being used to identify valuable metrics, weeding-out weak or problematic metrics is an equally worthwhile objective, especially if you subscribe to the view that the organization should stick with 'a few good metrics'. The method provides a rational, even-handed basis on which to analyze, compare and contrast metrics, then select or discard them. You might think of it as a quality filter for security metrics.]