Resilience as a business continuity mindset

An article written in conjunction with Dejan Kosutic has just been published at ContinuityCentral.com. 

"Most business continuity experts from an IT background are primarily, if not exclusively, concerned with establishing the ability to recover failed IT services after a serious incident or disaster. While disaster recovery is a necessary part of business continuity, this article promotes the strategic business value of resilience: a more proactive and holistic approach for preparing not only IT services, but also other business processes before an incident in order that an organization will survive incidents that would otherwise have taken it down, and so keep the business operating in some form during and following an incident."

We explain how resilience differs from and complements more conventional approaches to business continuity.  It is a cultural issue with strategic implications and benefits for everyday routine business, not just in crisis or disaster situations. It has implications throughout the organization, including business activities/processes, systems, workers and relationships with third parties. It is an integral and essential part of risk management.

The article discusses resilience in the context of ISO 22301 and ISO27k, and includes a maturity model and metric to help organizations put the strategy into practice.

Dejan and I share a passion for this topic that I hope comes across in our writing. Comments welcome!

Regards,

Gary (Gary@isect.com)

Awareness paper on authentication and phishing metrics

We've just republished a management-level security awareness paper on metrics relating to user authentication and phishing.

The introduction asks "How do we tell whether our authentication controls are effective?" and "What does 'effective' even mean in this context?" - two decent questions that could be addressed through suitable metrics.

Questions like these are central to the GQM (goal-question-metric) method (see IT Security Metrics by Lance Hayden), and not just literally in terms of their position in the handy acronym. They link the organization's goals or objectives relating to information security, to the information security metrics that are worth measuring.

In your particular circumstances, the effectiveness of authentication controls might or might not be of sufficient concern to warrant generating the associated metrics. Other aspects might take precedence, for example the amount invested in authentication controls, and the ongoing operating and maintenance costs of those controls. It's usually not too hard to think up a whole raft of aspects, parameters or concerns relating to the topic area, but focusing on the things that are likely to matter most to the organization (business priorities) is a good way to keep the list within reasonable bounds. Once you know what they are, the next step is to figure out the questions arising e.g. "Are we spending appropriately (neither too much nor too little) on authentication?"

From there, it's simply a matter of deciding what data would help address the questions, and those are your metrics!  Job done!  Errr, well, no, not quite: if you have several goals/areas of concern and numerous questions arising, each requiring multiple metrics to generate the answers, there is a distinct risk of being overwhelmed with possibilities. It is infeasible and in fact counterproductive to attempt to measure everything. Less is more! This is where the PRAGMATIC method comes into play as a way to whittle down the long list to a shortlist of metrics showing the most promise. The GQM approach also suggests filtering out the metrics that don't address the questions very well, and trimming down on metrics addressing questions that are only marginally related to the organization's business goals. Both approaches have their merits.

3 more metrics papers

We've just published another three documents on security metrics, written and first released five years ago as part of the management stream in the NoticeBored information security awareness service.

The first paper was concerned with measuring integrity.  Despite being one of the three central pillars of information security, integrity is largely overshadowed by availability and, especially, confidentiality ... and yet, if you interpret 'integrity' liberally, it includes some extremely important information security issues. The 'completeness and correctness' angle is pretty obvious, while 'up to date-ness' and 'appropriateness' are less well appreciated.  Add in the character and trustworthiness of people, and integrity takes on a rather different slant (Bradley Manning, Julian Assange and Edward Snowden springing instantly to mind as integrity failures).  An 'honesty metric' is an innovative idea.

The integrity metrics paper also suggests measuring the integrity of the organization's security metrics program or system of measurements, on the basis that metrics ought to be accurate, complete, up-to-date and relevant. The metrics integrity issue is obvious when you think about it. Managing with poor quality information is less than ideal.  However, in our experience, information security metrics are mostly taken at face value: we usually focus on what the numbers are telling us without even considering that they might perhaps be wrong, misleading, incomplete or inconsequential. Worse still, we get so distracted by the fancy "infographics" that the information content is almost irrelevant.  That's hardly a scientific approach!  We have raised this issue before in relation to treating published security surveys as gospel, blythely ignoring the fact that most are statistically dubious if not patently biased marketing copy. Remember this the next time you search the web for pie charts to illustrate your security investment proposals, or the next time someone tries to persuade you to loosen the purse strings! 

A short, humdrum paper on IT audit metrics suggests a few ways to measure the IT audit function, such as "IT audit program coverage" as well as conventional management metrics.  

The third paper on malware metrics was virtually the same as the version released a year earlier. We made some changes the following year, partly due to the research and thinking that went into writing PRAGMATIC Security Metrics ... but you'll have to wait just a bit longer for the 2009 paper.

Management without metrics - how?

The SEC (Security Executive Council - not the Securities and Exchange Commission!) boldly describes itself as "the leading research and advisory firm that specializes in security risk mitigation."  Their primary interest appears to be physical security, although they also make the odd nod towards IT security, business continuity and 'convergence'.

The SEC conducted an unscientific online poll, asking respondents to self-assess and report the capability maturity of their security programs using the classic 5 point SEI-CMM scale.  Unsurprisingly, the results show a vaguely normal distribution about the middle value ('defined'), skewed towards the low end of the maturity scale.

It appears they may have asked a separate question about metrics:

"When participants were asked about metrics (a higher level of maturity), 64% said they did not use business value metrics (metrics that are beyond initial "counting" of activities such as number of background checks performed or number of badges issued)."

So only about a third of their respondents have security metrics other than the absolute basics - a pathetically low proportion that begs the obvious question "How are they managing security without metrics?"

Answers on a postcard please.  Or comment below.

Metrics matter (updated)

An article by Mintz Levin about the 2013 privacy breach/information security incident at US retailer Target stated that the company has disclosed gross costs of $252 million, with some $90m recovered from its insurer leading to a net cost of $162m, up to the end of 2014 anyway (the incident is not over yet!).

Given that the breach apparently involved personal information on about 40 million people, it's trivial to work out that the incident apparently cost Target roughly $6 per compromised record ($4/record net of insurance payouts) ... but before anyone runs amock with those headline numbers, let's delve a bit deeper.

First off, what confidence do we have in the numbers themselves? The article cites its sources as 8-K filings, in other words Target's official reports concerning the incident to the Securities and Exchange Commission. Personally, I'm quite happy with that: the dollar amounts are not mere speculation but (I believe, not being a specialist in US laws and regulations) have been carefully drawn up, audited and formally approved by management - specifically Target's CFO. We could pore over the filed 8-K reports to verify them since they are published on the SEC site, or we could simply accept Mintz Levin's word: they are a law firm so there's a degree of trust and confidence. 

I took the 40 million compromised record count from a convenient web page somewhere - not as easy to verify but there are many such pages reporting similar numbers, so let's assume the figure is based on a count and disclosure by Target.  And let's assume it's correct (yes, another assumption).

Now dig further. Having tried to track and calculate the financial costs from relatively small information security incidents myself, I appreciate just how tough that can be in practice. The costs fall across two main categories, direct and indirect or consequential. The direct costs are a bit of a nightmare to monitor when everyone is running around frantically dealing with the incident at its height, but they can be estimated retrospectively and tracked fairly accurately once things calm down: it's a matter of cost accounting. Simply stated, someone assigns the direct expenses associated with the incident to an accounting code for the incident, and the financial system tots up and spews out the numbers. There are several opportunities for substantial error in there (for instance, signficant costs wrongly coded or neglected, and investments in information security/privacy improvements that would have been made anyway, regardless of the incident, being charged against it in order to secure the budgets and inflate the insurance claims), but these errors pale into insignificance against the indirect or consequential costs ...

A serious information security incident that becomes public knowledge seems likely to have an adverse impact on the organization's image and hence its brand values, but how much of an effect, in dollar terms? It's almost impossible to say with any certainty. In the case of a major incident, the company's marketing and financial people could evaluate and estimate the effects using metrics such as customer footfall, turnover, profitability, market surveys and so forth ... but potentially there is a conflict of interest there since those self-same people are charged with maintaining or boosting the company's brands and value, hence they may be understandably reluctant to report bad news to management. Furthermore, there are no easy, generally-accepted, accurate or independently-verifiable ways to convert changes in most of these metrics (such as "brand recognition") into dollars without a great deal of argument and doubt.

On top of that, there is some truth to the saying that "There's no such thing as bad news". Publicity of incidents is also publicity for the organizations and individuals involved. Publicity equates to media exposure and brand recognition hence, paradoxically, bad incidents might actually benefit those involved.

That leads us to consider stock price as another possible measure of the gross effects of an incident, one that conveniently enough is already in dollars and is widely reported, with historical data just a few click away (e.g. see the 3-year Target share price graph to the left here, courtesy of those nice people at MarketWatch.com). Given the number of shares issued (requiring a few more clicks), it's not too hard to convert the share price at any point into a market capitalization value for the company, and thus to calculate the effect the incident had on that value, but now it gets really interesting. After the incident was initially disclosed and widely reported in 2013, Target's share price declined markedly and then recovered in 2014, and is now well above the 2013 peak. What relation does that have to the incident? Again, it's almost impossible to say because there are just so many factors involved: stockbrokers, dealers and investors take a professional interest in identifying, evaluating and predicting those factors, and some of them are very successful so you might try asking them about the incident, but don't be surprised if they confuse you with statistics while keeping their trade secrets to themselves!

The same issue cropped up in the Sony hack at the end of last year. Sony's share price (plotted on the right over the past 6 months) has moved quite consistently upwards. There was a noticeable dip around the year end but it pretty much recovered its original trajectory by the end of January. I'm quite sure I could fit a straight line trend to the data with little statistical variance. 

OK, is that all there is to it? Well, no, we're not finished yet, not by a long chalk. 

So far we've only considered Target's costs: what about those whose personal information was disclosed, and the banks and other companies who have lost out to identity fraud? How much has the incident as a whole cost? How on Earth can we measure or calculate that? Once again, the short answer is that we can only estimate at best. 

What price would YOU put on the personal aggravation and grief caused by discovering that YOUR privacy has been breached and you may be the victim of identity theft? Go ahead, think about it and name your price! If enough of us did so, we might generate some sort of mean value but it's obviously highly subjective and doubtless extremely sensitive to the context and the precise questions we pose - plus of course there's the issue of our sampling strategy and sample size, since we can't ask everyone. Unfortunately, even a small error in our per-victim cost estimate will be massively amplified if we multiply that by the 40 million, so we really ought to take more care over this if the numbers matter - which they surely do as we'll come on to in a moment.

First, though, consider that the relationship between the total cost of a privacy breach/incident and the number of records disclosed is generally implied, but that is another unproven and potentially highly misleading assumption. We don't actually know the nature of the relationship, and it is likely to vary according to a number of factors aside from just the number of records. Identities belonging to the rich and famous are probably worth much more to identity thieves than those belonging to the the poor, for example, so a breach involving data from high-worth individuals, organizations or celebrities seems likely to result in greater losses than one involving the same number of records for "ordinary" people. Different items or types of information vary markedly in their inherent value (e.g. contrast the value of someone's email address or phone number to their credit card number - and then consider the additional value to fraudsters of obtaining multiple items in linked records). One might argue on basic arithmetic that the per-record costs decrease exponentially as the number of records increases, or that the relationship is non-linear due to the additional impact of news headlines with nice round figures ("more than 40 million" is worse than "almost 40 million", and far worse than "40 thousand"!). 

In privacy breaches, the black-market price of credit card numbers etc. is sometimes used to estimate the overall costs (e.g. if 'the average' record is worth, say, $2 to criminals, then $40m records are worth $80m). That simplistic approach begs various questions about how we determine the black-market price (which, by its very nature, is not openly available information), and at what point we measure it (since the value of stolen credit card numbers declines quite rapidly as word about the incident spreads and victims, banks and credit card companies progressively identify and cancel the cards). Furthermore, the costs accruing to the victims (i.e. Target and its owners/stakeholders, the data subjects, the banks and other institutions involved, oh and the FBI, police etc.) as a result of the incident may be related to but almost certainly exceed the profits accruing to the identity thieves, fraudsters and assorted middle-men exploiting it. Society as a whole picks up the discrepancies in a diffuse fashion.

That brings us to our final issue. Who cares how much infosec incidents such as this actually cost anyway? It matters because the information gets used in all sorts of ways, for example to justify investment in information security and privacy controls, incident management, insurance premiums, identity theft cover, contingency sums and more. It gets used for budgeting and benchmarking, for policy- and law-making. It feeds into our general appreciation of the information risks associated with personal information, and information risks as a whole. 

Stepping back a pace or two, this whole issue could be considered the elephant in the room for information risk and security professionals. We put enormous effort into promoting and justifying investments in information security controls to reduce the probability of, and damage caused by, incidents, trying our level best to persuade management to take heed of our concerns, support our business cases and invest adequately in security, especially proactive measures, systematic approaches and good practices such as ISO27k ... but if we look coldly and dispassionately at the situation including the assumptions and arguments laid out above, it could be said that incidents are not nearly as bad as we tend to make out, in other words we are crying wolf.  

Oh oh!  I guess we ought to firm up some of those estimates and assumptions, pronto, before we all lose our jobs! Metrics do matter, in fact.

PS The 2015 Verizon Data Breach Investigation Report attempts to define the mathematical relationship between 'Payout' and 'Records Lost' in so-called data breach incidents (see figure 21 and associated text), but acknowledges that although they have improved their model, they still don't have a firm grasp of all the relevant factors. Perhaps this blog piece will prompt them to re-evaluate their assumptions and presumptions, maybe even to do the research given the data and other resources available to them. Don't hold your breath though. I fully expect the mythical linkage between incident costs and records compromised to persist for many years yet, despite my best efforts. Its the infosec equivalent of the search for the holy grail - the Monty Python version. 

Comparative security metrics

In situations where it is infeasible or impracticable to quantify something in the form of a discrete count or a value in specific units, comparative or relative measures are a useful alternative. They are better than not measuring at all, and in some cases easier to comprehend and more useful in a practical sense. In this respect, we disagree with those in the field who fervently insist that all metrics must be expressed as numbers of units (e.g. "20 centimetres"). It seems to us "A bit longer than a pencil", while obviously imprecise, might be a perfectly legitimate and helpful measure of something (regardless of what that thing might be - a cut on your arm for instance).

Cardinal numbers and units of measure have their place, of course, but so do ordinals, comparatives and even highly subjective measures - all the way down to sheer guesswork (and, yes, 'down to' itself implies a comparative value). Douglas Hubbard's "How To Measure Anything" is an excellent, throught-provoking treatise on this very subject.

In information security, comparisons or relations can provide answers to entirely valid and worthwhile questions such as:

• Are we more or less secure than our peers?
• Are we getting more or less secure over time?
• If we both sustain our present rate of change, how long will it be before we'll surpass our competitors' level of information security?
• Are our information risks increasing or decreasing?
• Which are our strongest and weakest areas or aspects of security?
• Of all the myriad changes currently occuring in information security, what are the most worrying trends?
• Does information risk X fall within or exceed our risk appetite or tolerance?
• Which business unit, function, department or site is the most/least vulnerable?
• Are we spending too little, about the right amount, or too much on information security

As part of an information security awareness case study on 'the Sony hack', a management discussion paper describes three types of comparative security metrics with several examples of each.

Management awareness paper on contingency metrics

Here's the next security awareness paper in the series, describing metrics relating to contingency and business continuity management.

"Measuring the effectiveness of contingency arrangements is a tough challenge, not least because (like insurance policies) we hope we will never need to use them. However it makes sense to measure our investment in contingency plans and preparations, and to confirm whether management is sufficiently confident in them, prior to enacting them as by that stage it will be too late."

Possible contingency metrics suggested in the paper include:

• RTO and RPO - classic disaster recovery metrics in their own right
• Resilience - measured by incidents
• Recovery - proportions of systems for which RTP/RPO are defined, tested and met
• Costs - easier to measure than benefits, and yet an uncommon metric in practice
• Management confidence - to what extent do managers believe in the congtingency arrangements?

There are many other possible metrics in this area.  What do you measure? Why?  What do your contingency or business continuity metrics tell you?