tag:blogger.com,1999:blog-72097075638954302692024-03-14T09:03:23.161+13:00Security MetametricsMeasure to improve information risk and security managementGaryhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.comBlogger209125tag:blogger.com,1999:blog-7209707563895430269.post-44864505252853427242016-07-25T11:54:00.000+12:002016-12-19T08:47:27.435+13:00Blog merged into NBlog<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">To cut down on duplication and administration, I have decided in future to blog on security metrics in my main information security blog, <a href="http://blog.noticebored.com/" target="_blank">NBlog</a> (the <a href="http://blog.noticebored.com/" target="_blank">NoticeBored blog</a>) ... so this will be the final post here on the Security Metametrics blog.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif; font-size: large;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">I have merged the previous metrics blog items into NBlog, and I will continue blogging on security metrics alongside information security, governance, compliance, risk, ISO27k <i>etc</i>. whenever inspiration coincides with the free time to express my thoughts. I'm still just as fascinated as ever by the topic</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif; font-size: large;"><br /></span></div>
<div style="text-align: justify;">
<a href="https://3.bp.blogspot.com/-PFDUvQyIWSM/V5VUtRqWHrI/AAAAAAAAB6Q/ZhwP-323LWQsTijgggAJKoVMLL0ea5UkQCLcB/s1600/Dolfin.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://3.bp.blogspot.com/-PFDUvQyIWSM/V5VUtRqWHrI/AAAAAAAAB6Q/ZhwP-323LWQsTijgggAJKoVMLL0ea5UkQCLcB/s320/Dolfin.jpg" width="181" /></a><span style="font-family: "verdana" , sans-serif; font-size: large;">If you'd like to continue reading this stuff, please update your bookmarks and blog aggregators to point at blog.noticebored.com </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif; font-size: large;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">If not, well ... <a href="https://en.wikipedia.org/wiki/So_Long,_and_Thanks_for_All_the_Fish" rel="nofollow" target="_blank">so long and thanks for all the fish</a>.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif; font-size: large;"><br /></span></div>
<div style="text-align: justify;">
<br /></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-17130761243803640672016-07-22T11:36:00.005+12:002016-07-22T11:36:44.930+12:00Micro vs. macro metrics<div style="text-align: justify;">
<a href="https://4.bp.blogspot.com/-zSxX4rEtAiY/V5FXNIDD-3I/AAAAAAAAB54/lyIiaMcZrwE1li8u9Whu6c7kY8QZngXWgCLcB/s1600/Scope.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="316" src="https://4.bp.blogspot.com/-zSxX4rEtAiY/V5FXNIDD-3I/AAAAAAAAB54/lyIiaMcZrwE1li8u9Whu6c7kY8QZngXWgCLcB/s400/Scope.jpg" width="400" /></a><span style="font-family: "verdana" , sans-serif;">Whereas "micro metrics" focus-in on detailed parts, components or elements of something, "macro metrics" pan out to give a broad perspective on the entirety. </span><br />
<i style="font-family: verdana, sans-serif;"><br /></i>
<i style="font-family: verdana, sans-serif;">Both </i><span style="font-family: "verdana" , sans-serif;">types of metric have their uses.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Micro metrics support low-level operational management decisions. Time-sheets, for example, are micro metrics recording the time spent on various activities, generating reports that break down the hours or days spent on different tasks during the period. This information <i>can </i>be used to account for or reallocate resources within a team or department or identify. Normally, though, its true purpose is to remind employees that they are being paid for the hours they work, or as a basis on which to charge clients. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Macro metrics, in contrast, support strategic big-picture management decisions. They enable management to "see how things are going", make course-corrections and change speed where appropriate. The metric "security maturity", for example, has implications for senior managers that are lost on lower levels of the organization. I have a soft spot for maturity metrics: they score strongly on the <a href="http://www.securitymetametrics.com/html/sampler.html" target="_blank">PRAGMATIC criteria</a>, enabling us to measure complex, subjective issues in a reasonably objective and straightforward fashion.</span><br />
<span style="font-family: "verdana" , sans-serif;"><br /></span>
<span style="font-family: "verdana" , sans-serif;">The sausage-machine metrics churned out automatically by firewalls, enterprise antivirus systems, vulnerability scanners and so forth are almost entirely micro metrics, intensely focused on very specific and usually technical details. T</span><span style="font-family: "verdana" , sans-serif;">here are vast <i>oceans </i>of security-related data.</span><span style="font-family: "verdana" , sans-serif;"> Lack of data is not a problem with micro metrics - quite the opposite.</span><br />
<div class="MsoPlainText" style="text-align: start;">
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Some security professionals are 'boiling the ocean' using big data analytics tools in an attempt to glean useful information from micro metrics but a key problem remains. When they poke around in the condensate, they don't really know what they're looking for. </span><span style="font-family: verdana, sans-serif;">The tendency is to get completely lost in the sea of data, constantly distracted by shiny things and obsessing about the data or the analysis ... rather than the information, knowledge, insight and wisdom that they probably should have gone looking for in the first place.</span></div>
</div>
<div class="MsoPlainText" style="text-align: start;">
<div style="text-align: justify;">
<span lang="EN-US" style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span lang="EN-US" style="font-family: "verdana" , sans-serif;">It's like someone stumbling around aimlessly in the dark, hoping to bump into a torch!<o:p></o:p></span></div>
</div>
<div class="MsoPlainText" style="text-align: start;">
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
</div>
<div class="MsoPlainText" style="text-align: start;">
<div style="text-align: justify;">
<span lang="EN-US" style="font-family: "verdana" , sans-serif;">Just as bad, when a respected/trusted metrics "expert" discovers a nugget and announces to the world "Hey look, something shiny!", many onlookers trust the finder and assume therefore that the metric must be Good, without necessarily considering whether it even makes sense to their organization, its business situation, its state of maturity, its risks and challenges and so forth ... hence they are distracted once more. </span><span style="font-family: verdana, sans-serif;">As if that's not enough, when others chime in with "Hey look, I've polished it! It's even shinier!", the distractions multiply. </span></div>
</div>
<div class="MsoPlainText" style="text-align: start;">
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
</div>
<div class="MsoPlainText" style="text-align: start;">
<div style="text-align: justify;">
<span lang="EN-US" style="font-family: "verdana" , sans-serif;">The bottom-up approach is predicated on and perpetuates the myth of Universal Security Metrics - a set of metrics that are somehow inherently good, generally applicable and would be considered good practice. "So, what should we be measuring in security?" is a <i>very </i>common naive question. Occasionally we see various well-meaning people (yes, including me) extolling the virtues of specific metrics, our pet metrics (maturity metrics in my case). We wax lyrical about the beauty of our pet metrics, holding them up to the light to point out how much thy gleam and glint. <o:p></o:p></span></div>
</div>
<div class="MsoPlainText" style="text-align: start;">
<div style="text-align: justify;">
<br /></div>
</div>
<div class="MsoPlainText" style="text-align: start;">
<div style="text-align: justify;">
<span lang="EN-US"><span style="font-family: "verdana" , sans-serif;">What we almost never do is explain, in any real detail, how our pet metrics help organizations achieve their objectives. We may describe how the metrics are useful for security management, or how they address risk or compliance or whatever, but we almost invariably run out of steam well before discussing how they drive the organization towards achieving its business objectives, except for a bit of vague hand-waving, cloud-like. </span><o:p></o:p></span><br />
<span lang="EN-US"><span style="font-family: "verdana" , sans-serif;"><br /></span></span>
<span style="font-family: verdana, sans-serif;">By their very nature, it is even harder to see how micro metrics relate to the organization's business objectives. They are deep down in the weeds. Macro metrics may be up at the forest canopy level but even they are generally concerned with a specific area of concern - information security in my case - rather than with the business.</span><br />
<span style="font-family: verdana, sans-serif;"><br /></span>
<span style="font-family: verdana, sans-serif;">I guess that's why I like the <a href="http://www.securitymetametrics.com/html/hayden.html" target="_blank">Goal-Question-Metric approach</a> so much. Being explicit about the organizaiton's goals, its business and other high-level objectives (<i>e.g.</i> ethical or social responsibility and environmental protection), leads naturally into designing macro metrics with a clear business focus or purpose. </span></div>
<div style="text-align: justify;">
<span lang="EN-US"><span style="font-family: "verdana" , sans-serif;"><br /></span></span></div>
<div style="text-align: justify;">
<span lang="EN-US"><span style="font-family: "verdana" , sans-serif;">Kind regards,</span></span></div>
<div style="text-align: justify;">
<span lang="EN-US"><span style="font-family: "verdana" , sans-serif;">Gary</span></span></div>
</div>
</div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-91752801339020057982016-06-28T19:15:00.000+12:002016-06-28T19:15:17.528+12:00ISO27k conference in San Francisco, end of Sept<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-rZEbm1Z93bc/V3Ijrn8XpWI/AAAAAAAABzE/_g9mvX9bzYIRO34OiQhk9icI8YG9CKEAACLcB/s1600/27k%2BConference.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-rZEbm1Z93bc/V3Ijrn8XpWI/AAAAAAAABzE/_g9mvX9bzYIRO34OiQhk9icI8YG9CKEAACLcB/s1600/27k%2BConference.jpg" /></a></div>
<div class="MsoPlainText" style="text-align: justify;">
<a href="http://iso27001.com/" style="font-family: Verdana, sans-serif;">27k:
Security Summit for the Americas</a><span style="font-family: Verdana, sans-serif;"> will cover security metrics in the
context of the </span><a href="http://www.iso27001security.com/" style="font-family: Verdana, sans-serif;" target="_blank">ISO/IEC 27000 Information Security Management Sytems standards</a><span style="font-family: Verdana, sans-serif;">. </span></div>
<div class="MsoPlainText" style="text-align: justify;">
<br /></div>
<div class="MsoPlainText" style="text-align: justify;">
<span lang="EN-US"><span style="font-family: Verdana, sans-serif;">It's a
2-day conference plus optional workshops the day before and training courses
afterwards, in the final week of September at a <a href="http://www.ssfconf.com/">smart purpose-built conference facility</a> on
the outskirts of San Francisco airport, not far beyond the boundary fence I
think. Standing speakers may need to duck, and shout.<o:p></o:p></span></span></div>
<div class="MsoPlainText" style="text-align: justify;">
<br /></div>
<div class="MsoPlainText" style="text-align: justify;">
<span lang="EN-US"><span style="font-family: Verdana, sans-serif;">There will be <a href="http://iso27001.com/?page_id=957" rel="nofollow" target="_blank">sessions</a> on:</span></span></div>
<div class="MsoPlainText" style="text-align: justify;">
</div>
<ul>
<li><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">ISO27k
basics</span></li>
<li><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">ISO27k
implementation</span></li>
<li><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">ISO27k
for cloud security</span></li>
<li><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">Integrating
ISO 22301 (business continuity) with ISO27k</span></li>
<li><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">ISO27k
metrics …</span></li>
</ul>
<br />
<div class="MsoPlainText" style="text-align: justify;">
<span lang="EN-US"><span style="font-family: Verdana, sans-serif;">and
more.<o:p></o:p></span></span></div>
<div class="MsoPlainText" style="text-align: justify;">
<br /></div>
<div class="MsoPlainText" style="text-align: justify;">
<span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Walt
Williams of Lattice, Richard Wilshire (ISO/IEC JTC1/SC27 project leader
for the total revamp of <a href="http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=64120">ISO/IEC
27004</a> on “Monitoring, measurement, analysis and evaluation” – publication
imminent), and Jorge Lozano from PwC are all presenting on metrics at the
conference, and FWIW me too. I’m hoping to persuade Krag to attend as well. <o:p></o:p></span></span></div>
<div class="MsoPlainText" style="text-align: justify;">
<br /></div>
<div class="MsoPlainText" style="text-align: justify;">
<span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Aside
from the conference sessions, it is lining up to be <i>The Place</i> for security
metrics newbies and wise old owls alike to put the world to rights during the coffee
breaks, maybe over a meal, and then inevitably at a nearby airport hotel bar
until the wee small hours. Should be a hoot.<o:p></o:p></span></span></div>
<div class="MsoPlainText" style="text-align: justify;">
<br /></div>
<div class="MsoPlainText" style="text-align: justify;">
<span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Join
us? <a href="http://iso27001.com/?page_id=78">Register</a> by Aug 15<sup>th</sup>
for the early-booking rate of $530 for the core conference. Hopefully
that leaves enough time to persuade the boss that it will be an <i>invaluable</i>
personal development opportunity. Essential. Unmissable. <o:p></o:p></span></span></div>
<div class="MsoPlainText" style="text-align: justify;">
<br /></div>
<div class="MsoPlainText" style="text-align: justify;">
<span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Priceless.</span></span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-27481564548125518422016-05-24T18:52:00.001+12:002016-05-24T18:52:19.062+12:00Fascinating insight from a graph<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Long-time/long-suffering readers of this blog will know that I am distinctly cynical if not scathing about published surveys and studies in the information security realm, most exhibiting substantial biases, severe methodological flaws and statistical 'issues'. Most of them are, to be blunt, unscientific worthless junk, while - worse still - many I am convinced are <i>conscious and deliberate attempts to mislead us</i>, essentially marketing collateral, fluff and nonsense designed and intended to coerce us into believing conjecture rather than genuine attempts to gather and impart actual, genuine facts that we can interpret for ourselves.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Integrity is as rare as rocking-horse poo in this domain. </span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Well imagine my surprise today to come across a well-written report on an excellent scientifically-designed and performed study - viz "<a href="https://morningconsult.com/wp-content/uploads/2016/04/Tanium-Cybersec-Report.pdf" rel="nofollow" target="_blank">The accountability gap: cybersecurity & building a culture of responsibility</a>", a study sponsored by Tanium Inc. and Nasdaq Inc. and conducted by a research team from <a href="http://www.gold.ac.uk/" target="_blank">Goldsmiths</a> - an historic institution originally founded </span><span style="font-family: Verdana, sans-serif;">in the nineeenth Centu</span><span style="font-family: Verdana, sans-serif;">ry </span><span style="font-family: Verdana, sans-serif;">as the Technical and Recreative Institute </span><span style="font-family: Verdana, sans-serif;">for the Worshipful Company of Goldsmiths, one of the most powerful of London’s City Livery Companies</span><span style="font-family: Verdana, sans-serif;">. </span><span style="font-family: Verdana, sans-serif;">The Goldsmiths Institute mission was ‘the promotion of the individual skill, general knowledge, health and wellbeing of young men and women belonging to the industrial, working and poorer classes’. </span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">"Goldsmiths" (as it is known) is now a college within the University of London, based in Lewisham, a thriving multicultural borough South East of the City, coincidentally not far from where I used to work and live. I think it's fair to equate 'tradition' with 'experience', a wealth of culture, knowledge and expertise that transcends the ages.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">I'm not going to attempt to summarize or comment on the entire study here. Instead I restrict my commentary to a single graph, screen-grabbed from the report out of context, hopefully to catch your imagination as it did mine:</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-NdyeSWPvIjE/V0PpxaiN7gI/AAAAAAAABvs/XNw6WfcSXgM49D0_haF8fGzXmxVn1PKRwCLcB/s1600/Awareness%2Breadiness%2Bcorrelation.pdf.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-NdyeSWPvIjE/V0PpxaiN7gI/AAAAAAAABvs/XNw6WfcSXgM49D0_haF8fGzXmxVn1PKRwCLcB/s1600/Awareness%2Breadiness%2Bcorrelation.pdf.jpg" /></a></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">That scatter-graph clearly demonstrates the relationship between 'awareness' (meaning the level of cybersecurity awareness determined by the study of over 1,500 qualified respondents - mostly CISOs and non-exec directors plus other senior managers at sizeable UK, US, Japanese, German and Nordic organizations with at least 500 employees) and 'readiness' (essentially, their state of preparedness to repulse and deal with cybersecurity incidents). It is so clear, in fact, that statistics such as correlation are of little value.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">In simple terms, organizations that are aware are ready and face medium to low risks (of cybersecurity incidents) whereas those that are neither aware nor ready are highly vulnerable.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Even a correlation as strong and convincing as that does not formally <i>prove </i>a cause-effect relationship between the factors, but it certainly supports the possibility of a mechanistic linkage. It doesn't indicate whether cybersecurity awareness leads or lags readiness, for instance, but let's just say that I have my suspicions. In reality, it doesn't particularly matter.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><i>Please </i>download, read and mull-over the report. You might learn a thing or two about cybersecurity, and hopefully you'll see what I mean when I contrast the Goldsmiths study with the gutter-tripe we are normally spoon-fed by a large army of marketers, press releases, journalists and social networking sites.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Take a long hard look at the methodology, especially Appendix B within which is the following frank admission:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">"Initial examination of the responses showed that three of the Awareness questions were unsatisfactory statistically. (The
three related problems were that they did not make a satisfactory contribution to reliability as measured by Cronbach’s
alpha; they did not correlate in the expected direction with the other answers; and in at least one case, there was evidence
that it meant diferent things to diferent respondents.) With these three questions removed, the Awareness and Readiness
questions showed satisfactory reliability (as measured by Cronbach’s alpha)." </span></blockquote>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><a href="https://en.wikipedia.org/wiki/Cronbach%27s_alpha" rel="nofollow" target="_blank">Cronbach's (alpha)</a> is a statistical measure using the correlation or covariance between factors across multiple tests to identify inconsistencies. The team used it to identify three questions whose results were inconsistent with the remainder. Furthermore, they used the test in part to exclude or ignore particular questions, thereby potentially warping the entire study since they did not (within the report) fully explain <i>why </i>nor <i>how far </i>those particular questions were out of line, other than an obtuse comment about differences of interpretation in at least one case. In scientific terms, their exclusion was a crucial decision. Without further information, it raises questions about the method, the data and hence the validity of the study. On the other hand, the study's authors 'fessed up, explaining the issue and in effect asking us to trust their judgement as the original researchers, immersed in the study and steeped in the traditions of Goldsmiths. The very fact that they openly disclosed this issue immediately sets them apart from most other studies that end up in the general media, as opposed to the peer-reviewed scientific journals where such honest disclosures are <i>de rigeur</i>.</span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">I'd particularly like to congratulate Drs Chris Brauer, Jennifer Barth and Yael Gerson and team at Goldsmiths Institute of Management Studies, not just for that insightful graph but for a remarkable and yet modest, under-stated contribution to the field. Long may your rocking horses continue defecating :-)</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-18492122454194840772016-03-23T16:06:00.001+13:002016-03-23T16:06:28.079+13:00Another vendor survey critique<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">I've just been perusing another vendor-sponsored survey report - specifically the <a href="https://www.barkly.com/cybersecurity-confidence-report-2016" rel="nofollow" target="_blank">2016 Cybersecurity Confidence Report from Barkly</a>, a security software company. </span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">As is typical of marketing collateral, the 12 page report is strong on graphics but short on hard data. In particular, there is no equivalent of the 'materials and methods' section of a scientific paper, hence we don't know how the survey was conducted. They claim to have surveyed 350 IT pro's, for instance, but don't say how they were selected. Were they customers and sales prospects, I wonder? Visitors to the Barkly stand at a trade show perhaps? Random respondents keen to pick up a freebie of some sort for answering a few inane questions? An online poll maybe?</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The survey questions are equally vague. Under the heading "What did we ask them", the report lists:</span></div>
<ul>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Biggest concerns</b> [presumably in relation to cybersecurity, whatever that means];</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Confidence in current solutions,
metrics, and employees </b>[which appears to mean confidence in current cybersecurity products, in the return on investment for those products, and in (other?) employees. 'Confidence' is a highly subjective measure. Confidence in comparison to what? What is the scale?];</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Number of breaches suffered
in 2015 </b>[was breach defined? A third of respondents declined to answer this, and it's unclear why they were even asked this]</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Time spent on security </b>[presumably sheer guesswork here]</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Top priorities</b> [in relation to cybersecurity, I guess]</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Biggest downsides to
security solutions</b> [aside from the name! The report notes 4 options here: slows down the system, too expensive, too many updates, or requires too much headcount to manage. There are <i>many</i> more possibilities, but we don't know whether respondents were given free rein, offered a "something else" option, or required to select from or rank (at least?) the 4 options provided by Barkly - conceivably selected on the basis of being strengths for their products, judging by their strapline at the end: "At Barkly, we believe security shouldn’t be difficult to use or
understand. That’s why we’re building strong endpoint protection
that’s fast, affordable, and easy to use"].</span></li>
</ul>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Regarding confidence, the report states:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">"The majority of the respondents we surveyed struggle to determine the direct effect
solutions have on their organization’s security posture, and how that effect translates into
measurable return on investment (ROI). The fact that a third of respondents did not have the ability to tell whether their company
had been breached in the past year suggests the lack of visibility isn’t confined to ROI. Many companies still don’t have proper insight into what’s happening in their organization
from a security perspective. Therefore, they can’t be sure whether the solutions they’re
paying for are working or not."</span></blockquote>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">While I'm unsure how they reached that conclusion from the survey, it is an interesting perspective and, of course, a significant challenge for any company trying to sell 'security solutions'. I suspect they might have got better answers from execs and managers than from lower-level IT pro's, since the former typically need to justify budgets, investments and other expenditure, while the latter have little say in the matter. The report doesn't say so, however.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<a href="https://3.bp.blogspot.com/-CmBjJ0spr5U/VvIBScdjlJI/AAAAAAAABlU/uoSRUxoUEvwAJuVTxp8hOlZWKyDAGxzyA/s1600/Barkley%2Bsurvey.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: Verdana, sans-serif;"><img border="0" height="503" src="https://3.bp.blogspot.com/-CmBjJ0spr5U/VvIBScdjlJI/AAAAAAAABlU/uoSRUxoUEvwAJuVTxp8hOlZWKyDAGxzyA/s640/Barkley%2Bsurvey.gif" width="640" /></span></a></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Elsewhere the report does attempt to contrast responses from IT pro's (two-thirds of respondents, about 230 people) against responses from IT executives and managers (the remaining one-third, about 120) using the awkwardly-arranged graphic above. The associated text states:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">"When our survey results came in, we quickly noticed a striking difference in
attitudes among IT professionals in non-management positions and their
counterparts in executive roles. These two groups responded differently to
nearly every question we asked, from time spent on security to the most
problematic effect of a data breach. Stepping back and looking at the survey
as a whole, one particular theme emerged:
When it comes to security, executives are much more confident than
their IT teams."</span></blockquote>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Really? Execs are "much more confident"? There is maybe a little difference between the two sets of bars, but would you call it 'much' or 'striking'? Is it statistically significant, and to what confidence level? Again we're left guessing.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<h3 style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Conclusion</span></h3>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">What do you make of the report? Personally, I'm too cynical to take much from it. It leaves far too much unsaid, and what it does say is questionable. Nevertheless, I would not be surprised to see the information being quoted or used out of context - and so the misinformation game continues.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">On a more positive note, the survey has provided us with another case study and further examples of what-not-to-do.</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-69792785495447485542016-03-19T15:01:00.000+13:002016-07-23T20:02:28.878+12:00How effective are our security policies?<div class="MsoNormal" style="text-align: justify;">
<a href="https://3.bp.blogspot.com/-p2r8UIsREko/VuzNLLszcLI/AAAAAAAABlE/zkm5rzTIoxUMCndTV7752vq1ZfBBQq_Yw/s1600/Network%2Bsecurity%2Bpolicy%2Bposter%2B350.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-p2r8UIsREko/VuzNLLszcLI/AAAAAAAABlE/zkm5rzTIoxUMCndTV7752vq1ZfBBQq_Yw/s1600/Network%2Bsecurity%2Bpolicy%2Bposter%2B350.jpg" /></a><span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;"><a href="https://groups.google.com/forum/#!topic/iso27001security/94m_Vz184FY" target="_blank">On the ISO27k Forum today</a>, someone asked us (in not so many words) how to determine or prove that the organization's information security policies are effective. Good question!</span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;"><br /></span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;">As a consultant working with lots organizations over many years, I've noticed that the quality of their information security policies is generally <i>indicative</i> of the maturity and quality of their approach to information security as a whole. In metrics terms, it is a security indicator.</span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;"><br /></span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;">At one extreme, an organization with rotten policies is very unlikely to be much good at other aspects of information security - but what exactly do I mean by 'rotten policies'? </span></span><span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;">I was thinking of policies that are badly-written, stuffed with acronyms, gobbledegook and often pompous or overbearing pseudo-legal language, with gaping holes </span></span><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -24px;">regarding current information risks and security controls, internal inconsistencies, out-of-date <i>etc.</i></span><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -24px;"> </span><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">... but there's even more to it than their inherent quality since policies <i>per se </i>aren't self-contained controls: they need to be </span><i style="color: #002060; font-family: Verdana, sans-serif; text-indent: -18pt;">used </i><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">which in practice involves a bunch of other activities.</span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;"><br /></span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;">At the other extreme, what would constitute excellent security policies? Again, it's not just a matter of how glossy they are. Here are some the key criteria that I would say are indicative of effective policies:</span></span></div>
<div class="MsoNormal" style="text-align: justify;">
</div>
<ul>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">The policies truly
reflect management’s intent: management understands, supports and
endorses/mandates them, and (for bonus points!) managers overtly comply with
and use them personally (they walk-the-talk);</span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">They also reflect
current information risks and security requirements, compliance obligations,
current and emerging issues <i>etc</i>. (<i>e.g.</i> cloud, BYOD, IoT and ransomware for
four very topical issues);</span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">They cover all
relevant aspects/topics without significant gaps or overlaps (especially no
stark conflicts);</span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">They are widely
available and read … implying also that they are well-written, professional in
appearance, readable and user-friendly;</span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">People refer to them
frequently (including cross-references from other policies, procedures <i>etc</i>.,
ideally not just in the information risk and security realm);</span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">They are an integral
part of security management, operational procedures <i>etc.;</i></span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">They are used in and
supported by a wide spectrum of information security-related training and awareness activities;</span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">Policy compliance is
appropriately enforced and reinforced, and is generally strong;</span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -18pt;">They are proactively
maintained as a suite, adapting responsively as things inevitably change;</span></li>
<li style="text-align: justify;"><span style="color: #002060; text-indent: -18pt;"><span style="font-family: "verdana" , sans-serif;">Users (managers,
staff, specialists, auditors and other stakeholders) value and appreciate them,
speak highly of them <i>etc</i>.</span></span></li>
</ul>
<div style="text-align: justify;">
<span style="color: #002060; font-family: "verdana" , sans-serif; text-align: justify;">As </span><span style="color: #002060; font-family: "verdana" , sans-serif; text-align: justify;">I'm about to conduct an ISO27k gap analysis for a client, I'll shortly be turning those criteria into a maturity metric of the type shown in appendix H of </span><a href="http://tinyurl.com/PRAGMATICmetrix" rel="nofollow" style="font-family: verdana, sans-serif;" target="_blank">PRAGMATIC Security Metrics</a><span style="color: #002060; font-family: "verdana" , sans-serif; text-align: justify;">. The approach involves documenting a range of scoring norms for a number of relevant criteria, developing a table to use as a combined checklist and measurement tool. Taking just the first bullet point above, for instance, I would turn that into 4 scoring norms roughly as follows:</span></div>
<ul>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif;"><b>100% point</b>: "</span><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -24px;">The policies truly reflect management’s intent: management full understands, supports and endorses/mandates them, managers overtly comply with and use them personally, and <i>insist </i>on full compliance";<br /></span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -24px;"><b>67% point</b>: "Managers formally mandate the policies but there are precious few signs of their genuine support for them: they occasionally bend or flaunt the rules and are sometimes reluctant to enforce them";<br /></span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -24px;"><b>33% point</b>: "Managers pay lip-service to the policies, sometimes perceiving them to be irrelevant and inapplicable to them personally and occasionally also their business units/departments, with compliance being essentially optional";<br /></span></li>
<li style="text-align: justify;"><span style="color: #002060; font-family: "verdana" , sans-serif; text-indent: -24px;"><b>0% point</b>: "Managers openly disrespect and ignore the policies. They tolerate and perhaps actively encourage noncompliance with comments along the lines of 'We have a business to run!'"</span></li>
</ul>
<div>
<div class="MsoNormal" style="text-align: justify;">
</div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; font-family: "verdana" , sans-serif;">During the </span><span style="color: #002060; font-family: "verdana" , sans-serif; text-align: justify;"> gap analysis, I'll systematically gather and review relevant evidence, assessing the client against the predefined norms row-by-row to come up with scores based partly on my subjective assessment, partly on the objective facts before me. The row and aggregate scores will be part of my closing presentation and report to management, along with recommendations where the scores are patently inadequate (meaning well below 50%) or where there are obvious cost-effective opportunities for security improvements (low-hanging fruit). What's more, I'll probably leave the client with the scoring table, enabling them to repeat the exercise at some future point </span><i style="color: #002060; font-family: verdana, sans-serif;">e.g. </i><span style="color: #002060; font-family: "verdana" , sans-serif; text-align: justify;">shortly before their certification audit is due and perhaps annually thereafter, demonstrating hopefully their steady progress towards maturity.</span><br />
<span style="color: #002060; font-family: "verdana" , sans-serif;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; font-family: "verdana" , sans-serif;">Regards,</span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="color: #002060; font-family: "verdana" , sans-serif;">Gary</span></div>
</div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-15856458467699226232016-02-25T12:04:00.000+13:002016-03-01T16:31:53.026+13:00CIS cyber security metrics<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The latest and greatest sixth version of the CIS (Center for Internet Security) <a href="https://www.cisecurity.org/critical-controls.cfm" rel="nofollow" target="_blank">Critical Security Controls</a> (now dubbed the "CIS Controls For Effective Cyber Defense") is supported by a <a href="https://www.cisecurity.org/critical-controls/download.cfm?f=A%20Measurement%20Companion%20to%20the%20CIS%20Critical%20Security%20Controls%20VER%206.0%2010.15.2015" rel="nofollow" target="_blank">companion guide to the associated metrics</a>. Something shiny in the introduction to the guide caught my beady eye:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><i>"There are lots of things that can be measured, but it is very unclear which of them are in fact worth
measuring (in terms of adding value to security decisions)."</i></span></blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Sounds familiar. In <a href="http://tinyurl.com/PRAGMATICmetrix" rel="nofollow" target="_blank">PRAGMATIC Security Metrics</a>, we said:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><i>"There is no shortage of ‘things that could be measured’ in relation to information security. Anything that changes can be measured both in terms of the amount and the rate of observable change, and possibly in other dimensions as well. Given the dynamic and complex nature of information security, there are a great number of things we could measure. It’s really not hard to come up with a long list of potential security metrics, all candidates for our information security measurement system. For our purposes, the trick will be to find those things that both (a) relate in a reasonably consistent manner to information security, preferably in a forward-looking manner, and (b) are relevant to someone in the course of doing their job, in other words they have purpose and utility for security management."</i></span></blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">From there on, though, we part company. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The CIS approach is highly prescriptive. They have explicitly identified and detailed very specific metrics <span style="text-align: justify;">for each of the recommended controls. For example, the metric associated with control 4.5:</span></span><br />
<blockquote class="tr_bq">
<span style="font-family: "verdana" , sans-serif;"><span style="text-align: justify;"><i>"</i></span><i>Deploy automated patch management tools and software update tools for operating system and software/applications on all systems for which such tools are available and safe. Patches should be applied to all systems, even systems that are properly air gapped."</i></span></blockquote>
<span style="font-family: "verdana" , sans-serif;">asks </span><br />
<blockquote class="tr_bq">
<span style="font-family: "verdana" , sans-serif;">"</span><i style="font-family: Verdana, sans-serif;">How long does it take, on average, to completely deploy
application software updates to a business system (by business
unit)?". </i></blockquote>
<span style="font-family: "verdana" , sans-serif;">To answer that particular question, three distinct values are suggested, <i>viz</i> 1,440, 10,080 or 43,200 minutes (that's a day, a week or a month in old money). It is <i>implied </i>that those are categories or rough guides for the response, so why on Earth they felt the need to specify such precise numbers is beyond me</span><span style="font-family: "verdana" , sans-serif;">. </span><span style="font-family: "verdana" , sans-serif;">Curiously, precisely the same three values are used in most if not all of the other suggested metrics relating to time periods ... which might be convenient but disregards the differing priorities/timescales likely in practice. I'd have thought some controls are rather more urgent than others. For instance, the time needed by the organization to restore normal IT services following a disaster is </span><i style="font-family: Verdana, sans-serif;">markedly </i><span style="font-family: "verdana" , sans-serif;">different to that required by an intrusion detection system to respond to a identified intrusion attempt. These are not even in the same ballpark.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The same concern applies to the CIS' proportional metrics. The suggested three choices in all cases are <i>"Less than 1%"</i>, <i>"1% to 4%"</i> or <i>"5% to 10%"</i>. </span><br />
<span style="font-family: "verdana" , sans-serif;"><br /></span>
<span style="font-family: "verdana" , sans-serif;">Note that for both types, answers above the maximum value are unspecified.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Note also that the response categories cover different ranges for those types of metric. The timescale values are roughly exponential or logarithmic, whereas the proportions are more linear ... but just as arbitrary. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Oh and the timescales are point values, whereas the proportions are ranges.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The only rationale presented in the paper for the values is this vagueness:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><i>"For each Measure, we present Metrics, which consist of three “Risk Threshold” values. These values represent an opinion from experienced practitioners, and are not derived from any specific empirical data set or analytic model. These are offered as a way for adopters of the Controls to think about and choose Metrics in the context of their own security improvement programs."</i></span></blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><span style="text-align: justify;">Aside from the curious distinction between measures and metrics, what are we to understand by 'risk thresholds'? Who knows? </span></span><span style="font-family: "verdana" , sans-serif;">They are hinting at readers adapting or customizing the values (if not the metrics)</span><span style="font-family: "verdana" , sans-serif;"> but I rather suspect that those who most value the CIS advice would simply accept their suggestions as-is.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span>
<span style="font-family: "verdana" , sans-serif;">Later in the metrics paper, the style of metrics changes to this:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><i>"CSC 1: Inventory of Authorized and Unauthorized Devices - Effectiveness Test. To evaluate the implementation of CSC 1 on a periodic basis, the evaluation team will connect hardened test systems to at least 10 locations on the network, including a selection of subnets associated with demilitarized zones (DMZs), workstations, and servers. Two of the systems must be included in the asset inventory database, while the other systems are not. The evaluation team must then verify that the systems generate an alert or email notice regarding the newly connected systems within 24 hours of the test machines being connected to the network. The evaluation team must verify that the system provides details of the location of all the test machines connected to the network. For those test machines included in the asset inventory, the team must also verify that the system provides information about the asset owner."</i></span></blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">As I said, this is a highly prescriptive approach, very specific and detailed on the measurement method. It's the kind of thing that <i>might </i>be appropriate for formalized situations where some authority directs a bunch of subserviant organizations, business units, sites or whatever to generate data in a standardized manner, allowing direct, valid comparisons between them all (assuming they follow the instructions precisely, which further implies the need for compliance activities).</span><br />
<span style="font-family: "verdana" , sans-serif;"><br /></span>
<span style="font-family: "verdana" , sans-serif;">Anyway, despite my criticisms, I recommend checking out the <a href="https://www.cisecurity.org/critical-controls.cfm" rel="nofollow" target="_blank">CIS critical controls for cyber defense</a>. Well worth contemplating.</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com2tag:blogger.com,1999:blog-7209707563895430269.post-74011527120978775952016-02-20T10:48:00.001+13:002016-02-20T10:52:10.274+13:00Zurich Insurance global cyber risk reports<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Zurich Insurance published <a href="http://knowledge.zurich.com/cyber-risk/cyber-costs-threaten-to-exceed-benefits/" target="_blank">a web page with a bunch of graphs projecting the global costs and benefits of cybersecurity under various scenarios</a> ... but what do they mean? What is the basis for analysis? I find the graphs confusing, almost devoid of meaning like so many infographics, a triumph of marketing gloss over substance. The page succeeded, however, in catching my beady eye.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Although Zurich neglected to provide a working hyperlink, Google led me inexorably to the research paper from which the graphs were plucked: <a href="http://www.atlanticcouncil.org/publications/reports/risk-nexus-overcome-by-cyber-risks-economic-benefits-and-costs-of-alternate-cyber-futures" target="_blank">Risk Nexus: Overcome by Cyber Risks? Economic Benefits and Costs of Alternate Cyber Futures</a> is a report by the Zurich Insurance Group and the Atlantic Council's Brent Scowcroft Center on International Security plus the Pardee Center for International Futures at the University of Denver, a follow-up to their 2014 report: <a href="http://www.atlanticcouncil.org/publications/reports/beyond-data-breaches-global-interconnections-of-cyber-risk" target="_blank">Beyond Data Breaches: Global Aggregations of Cyber Risk</a>. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Apart from casually referring to "cybers</span><span style="font-family: "verdana" , sans-serif;">pace" as 'the internet and associated IT', the reports are littered with undefined/vagu</span><span style="font-family: "verdana" , sans-serif;">e cyber terms such as "cyber risks", "cyber attacks", "cyber crime", "cyber incidents", "cyber shocks" and "cyber futures". </span><span style="font-family: "verdana" , sans-serif;">You might be comfortable with "cyber" but r</span><span style="font-family: "verdana" , sans-serif;">eplacing it with "Internet-related" suits me better since they are not talking about information or IT security in general, nor about cyberwar in particular - two other common cyber-interpretations.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<h3 style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The 2014 report</span></h3>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><a href="http://www.atlanticcouncil.org/images/publications/Zurich_Cyber_Risk_April_2014.pdf" rel="nofollow" target="_blank">The 2014 report</a> conjured up and considered a potential disaster scenario involving a major Internet-related incident at a large communications technology firm triggering cascading failures affecting the global economy, in other words a systemic risk with global repurcussions:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">"Early on, we nicknamed this project ‘cyber sub-prime’ because we intended it to expose the global aggregations of cyber risk as analogous to those risks that were overlooked in the U.S. sub-prime mortgage market. Problems in that segment spread far beyond the institutions that took the original risks, and proved severe enough to administer a shock that reverberated throughout the entire global economy. At first, the term ‘cyber sub-prime’ was just a quirky nickname, but it soon became a useful analogy, helping us to gain additional insights into cyber risks based on extended parallels with the financial sector."</span></blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">While there is value in drawing lessons from the global financial crisis, I wonder if maybe the research team has been blinkered into that particular mode of thinking or world view, ignoring other possible futures such as, say, terrorism or more gradual as opposed to </span><span style="font-family: "verdana" , sans-serif;">sudden crises, overpopulation for example? </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Anyway, t</span><span style="font-family: "verdana" , sans-serif;">he report recommended </span><span style="font-family: "verdana" , sans-serif;">"several concrete steps that must be taken to overcome these inevitable shocks of the future and prevent what could be called a 'cyber sub-prime' meltdown. </span><span style="font-family: "verdana" , sans-serif;">Recommendations to be resilient to cyber shocks include:</span></div>
<blockquote class="tr_bq">
<ul>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Putting the private sector at the center of crisis management, since government management of cyber risk lacks the agility needed</span></li>
</ul>
<ul>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Developing plans within organizations that have system-wide responsibility that ensure the stability of the system as a whole, rather than risks to an individual organization</span></li>
</ul>
<ul>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Creating redundant power and telecommunications suppliers and alternate ISPs connect to different peering points</span></li>
</ul>
<ul>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Investing in trained teams ready to respond with defined procedures</span></li>
</ul>
<ul>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Conducting simulations of the most likely and most dangerous cyber risks to better prepare"</span></li>
</ul>
</blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">I appreciate what they are getting at in the first bullet but I'm not sure I agree with it. The private sector <i>may </i>arguably be more 'agile' in managing Internet-related risks, but overall is it doing any better in fact? I see little evidence that the private sector is any more highly protected than the government sector, particularly given differences in the nature of their respective risks. Even if that's true, why did they ignore or discount the obvious strategic option of improving government sector Internet-related security, I wonder? Perhaps the fact that the research was funded by a private-sector insurance company has something to do with it ... </span></div>
<div>
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Their other points about considering systemic risk and developing more resilient infrastructures, effective incident response and training exercises involving simulations are fine by me, conventional and widely supported. </span><span style="font-family: "verdana" , sans-serif;">The possibility of <i>complete, permanent failure of the Internet </i>is but one of several extreme disaster scenarios that I recommend clients consider for information risk and business continuity management purposes. My key point is not to plan too narrowly for any <i>one </i>particular scenario (or in fact any of the unbounded set of credible situations that could lead to such an outcome, such as an all-out cyberwar) but to use <i>a wide variety </i>of diverse scenarios to </span><span style="font-family: "verdana" , sans-serif;">develop more comprehensive resilience, recovery and contingency arrangements in a far more general sense. Preparing for the worst case has benefits under less extreme conditions too, while there are far too many scary possibilities to risk being unprepared for what actually transpires.</span></div>
<div>
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">As to whether those five bullets constitute "concrete steps", I guess it's a matter of perspective or terminology. The report stops well short of providing pragmatic action plans and allocating responsibilities. Not so much rock-hard concrete as sloppy mud! [In contrast, take a look at the <a href="http://blog.noticebored.com/2015/08/lessons-from-aviation-industry.html" target="_blank">ICAO Global Aviation Safety Plan</a>, a strategic approach to ensure continued safety in the global aviation industry, laying out specific actions, responsibilities and timescales: now that's what I call concrete!]</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<h3 style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The 2015 report</span></h3>
<div>
<span style="font-family: "verdana" , sans-serif; text-align: justify;">The risk and economic modeling study evidently continued, leading to <a href="http://www.atlanticcouncil.org/cyberrisks/" rel="nofollow" target="_blank">last year's report</a>. I'll leave you to cast a cynical eye over the latest report. I'm too jaded to take it seriously.</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com2tag:blogger.com,1999:blog-7209707563895430269.post-44340945556604331812016-02-19T10:51:00.000+13:002016-02-20T11:18:25.570+13:00Security awareness metrics<div style="text-align: justify;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-21fG76pHpYI/VsY8dQj0-bI/AAAAAAAABkA/3xlUJqQk-QY/s1600/Awareness%2Bdashboard.gif" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="286" src="https://4.bp.blogspot.com/-21fG76pHpYI/VsY8dQj0-bI/AAAAAAAABkA/3xlUJqQk-QY/s400/Awareness%2Bdashboard.gif" width="400" /></a></div>
<span style="font-family: "verdana" , sans-serif;">Some say that information security awareness is hard to measure, and yet a moment's thought reveals several obvious, straightforward and commonplace metrics in this area, such as:</span></div>
<ul>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Attendance</b> numbers, trends, rates or proportions at awareness and training events;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Feedback </b>scores and comments from attendees at/participants in said events, or concerning other awareness activities, promotions, media, messages <i>etc.</i>; </span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">General, broad-brush, state-of-the-nation security awareness <b>surveys </b>of various populations or constituencies conducted on paper or using electronic forms or polls;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">More specific information recall and comprehension <b>tests </b>relating to awareness topics or sessions, conducted on paper or online (maybe through the Learning Management System);</span></li>
<li style="text-align: justify;"><b style="font-family: Verdana, sans-serif;">Awareness program metrics</b><span style="font-family: "verdana" , sans-serif;"> concerning activities planned and completed, topics covered (breadth </span><i style="font-family: Verdana, sans-serif;">and </i><span style="font-family: "verdana" , sans-serif;">depth of coverage), budget and expenditure ($ </span><i style="font-family: Verdana, sans-serif;">and </i><span style="font-family: "verdana" , sans-serif;">man-days), comparisons against other forms of security control and against other awareness programs (in other fields and/or other organizations). </span></li>
</ul>
<span style="font-family: "verdana" , sans-serif;"></span><br />
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">With a little more thinking time, it's quite easy (for me, anyway) to come up with a broader selection of awareness metrics also worth considering: </span></div>
<span style="font-family: "verdana" , sans-serif;">
</span>
<ul>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>More elaborate versions of the above</b>, perhaps combining metrics for more meaningful analysis - for instance using attendance records <i>and </i>feedback to compare the popularity and effectiveness of different types of awareness and training events, different topics, different timings, different presenters, different media <i>etc.</i>;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Page hit rates, stickiness and various other <b>webserver metrics </b>concerning the popularity of/interest in the information security intranet site, including various elements within it, such as the security policies and specific topic areas;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Metrics gleaned from <b>personnel records</b> (<i>e.g.</i> proportions of the workforce with basic, intermediate or advanced qualifications, or with skills and competencies relating to information security, privacy, governance, risk <i>etc</i>., and currency of their skills, knowledge, competencies and qualifications);</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>T</b></span><b style="font-family: Verdana, sans-serif;">argeted surveys/polls</b><span style="font-family: "verdana" , sans-serif;"> comparing and contrasting awareness levels between various groups (</span><i style="font-family: Verdana, sans-serif;">e.g. </i><span style="font-family: "verdana" , sans-serif;">different business units, departments, teams, levels, specialisms, ages, sexes, cultures/nationalities </span><i style="font-family: Verdana, sans-serif;">etc</i><span style="font-family: "verdana" , sans-serif;">.) or times (</span><i style="font-family: Verdana, sans-serif;">e.g.</i><span style="font-family: "verdana" , sans-serif;"> before, during and after specific awareness/training events, awareness focus periods, business periods </span><i style="font-family: Verdana, sans-serif;">etc.</i><span style="font-family: "verdana" , sans-serif;">) or topics (</span><i style="font-family: Verdana, sans-serif;">e.g. </i><span style="font-family: "verdana" , sans-serif;">phishing vs. other forms of social engineering, malware, fraud </span><i style="font-family: Verdana, sans-serif;">etc.)</i><span style="font-family: "verdana" , sans-serif;">;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Workforce security awareness/culture surveys and studies </b>conducted in person by trained and competent survey/research teams (a more expensive method that <i>can </i>generate better quality, richer, more valuable information);</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Maturity metrics</b> using audits, reviews, surveys and self-assessments to determine the maturity and quality of the organization's overall approach to security awareness and training relative to the state of the art in awareness (as documented in various standards, books and websites);</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Benchmarking</b> - comparing information security awareness levels, activities, spending <i>etc.</i> against other fields (such as health and safety or legal compliance) or organizations, industries <i>etc.</i>;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Risk-based awareness metrics</b>, perhaps assessing the relevance of employee awareness, understanding, knowledge, competence, responsiveness <i>etc.</i> to various information risks, issues or challenges facing the organization, giving a natural priority to the planned awareness and training topics and a basis for budgeting (including resourcing for the security awareness and training program);</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Risk-based information security metrics</b> looking at myriad sources to identify current information risks, trends, predictions, technology directions, emerging threats <i>etc</i>. (useful for strategic planning in information security, of course, with an obvious link through to the corresponding awareness and training needs);</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Change metrics</b> concerning change management and changes affecting the organization, especially those relevant to information risk, security, privacy <i>etc., </i>as well as measuring and driving changes within the awareness program itself<i>;</i></span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Process metrics</b> concerning various information risk, security, privacy, governance and compliance-related processes (again including those concerning awareness and training) and various parameters thereof (<i>e.g.</i> cost and effort, efficiency, effectiveness, consistency, complexity, compliance, creativity, risk ...); </span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Quality metrics </b>concerning the awareness content/materials including policies, procedures and guidelines: there are many possible parameters here <i>e.g. </i>the style of writing and graphics, professionalism, review and authorization status, breadth and depth of coverage, currency/topicality and relevance, readability (<i>e.g.</i> Flesch scores), interest/engagement levels, consistency;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Awareness surveys conducted by information security presenters, trainers and other professionals</b>: people attending training courses, conferences, workshops and so forth are generally accustomed to completing survey/feedback forms concerning the events <i>e.g. </i>the quality and competence of the presenter/trainer/facilitator, the materials, the venue, the catering <i>etc.</i> and, fair enough, that's quite useful information for the planners of such events. Why not also get the people who present/train/facilitate/lead the events to rate their audiences as well, on parameters such as interest in the topic, engagement, knowledge levels, receptiveness <i>etc.</i>? Your Information Security Management, Security Admin, Help Desk, PC Support, Risk and Compliance people will have a pretty good idea about awareness and competence levels around the organization. Management, as a whole, knows this stuff too, and so do the auditors ... so ask them!;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;"><b>Customer contact metrics</b> for the information security team including the security awareness people, measuring the nature and extent of their interactions with people both within and without the business (<i>e.g.</i> their attendance at professional meetings, conferences, webinars, courses <i>etc.</i>);</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Various awareness metrics gleaned from </span><b style="font-family: Verdana, sans-serif;">Help Desk/incident records</b><span style="font-family: "verdana" , sans-serif;"> relating to events and incidents reported (</span><i style="font-family: Verdana, sans-serif;">e.g. </i><span style="font-family: "verdana" , sans-serif;">mean time to report, as well as mean time to resolve, incidents), help requests (number and complexity, perhaps split out by business unit or department), issues known or believed to have been caused by ignorance/carelessness </span><i style="font-family: Verdana, sans-serif;">etc</i><span style="font-family: "verdana" , sans-serif;">., as well as general security metrics concerning incident rates for various types of information security incident - another driver to prioritize the planning and coverage of your awareness activities.</span></li>
</ul>
<span style="font-family: "verdana" , sans-serif;"></span><br />
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">I could continue but even <i>my </i>eyes are glazing over at this point, so instead I want to end with some quick comments about how to make sense of all those and other options, and how you might go about selecting 'a few good security awareness metrics' that might be worth actually using.</span></div>
<span style="font-family: "verdana" , sans-serif;">
</span>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Two specific approaches I recommend are <b>PRAGMATIC</b> and <b>GQM</b>. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><b>GQM </b>starts with some exploration and analysis of your organization's <b>goals</b> or strategic objectives for information risk, security, privacy, governance, compliance and all that jazz (especially how these aspects support or enable core business), leading to some fairly obvious high-level <b>questions</b> (<i>e.g. </i>"Are we sufficiently compliant with our legal obligations towards privacy?") and thence to the kinds of <b>metrics</b> that would generate the data that might address or answer those questions (privacy compliance metrics in that case). At a lower level of detail, the same approach can be used to determine the goals, questions and kinds of metrics for security awareness. [Sorry, I'm not going to do that for you - it's your homework for today!] [For more on GQM, read Lance Hayden's book <a href="http://tinyurl.com/hhayden" rel="nofollow" target="_blank">IT Security Metrics</a>].</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><b>PRAGMATIC </b>is a rational basis for choosing between a bunch of possible metrics and assorted variants, or to guide the creative development of new metrics, or to drive improvement by weeding out ineffective metrics and getting more value out of those that remain, using nine key criteria or parameters for metrics: <b>P</b>redictiveness, <b>R</b>elevance, <b>A</b>ctionability, <b>G</b>enuineness, <b>M</b>eaninfulness, <b>A</b>ccuracy, <b>T</b>imeliness, <b>I</b>ntegrity/<b>I</b>ndependence and <b>C</b>ost-effectiveness. [For more on PRAGMATIC, read our book <a href="http://tinyurl.com/PRAGMATICmetrix" rel="nofollow" target="_blank">PRAGMATIC Security Metrics</a>, browse this <a href="http://www.securitymetametrics.com/" target="_blank">website</a> or <a href="http://securitymetametrics.blogspot.co.nz/" target="_blank">blog</a>, or ask me!]</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-79662007432228484152016-02-15T17:30:00.002+13:002016-02-20T10:45:52.046+13:00We don't know, we just don't know UPDATED<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-7QE2luvoTp8/UacF1JGzReI/AAAAAAAAAcc/ptwAnNfCpGI/s1600/Cumul.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-7QE2luvoTp8/UacF1JGzReI/AAAAAAAAAcc/ptwAnNfCpGI/s1600/Cumul.gif" /></a></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Crime-related metrics are troublesome for several reasons. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Firstly, crime tends to be hidden, out of sight, mostly in the shadows. An unknown number of crimes are never discovered, hence recognized/identified incidents may not be representative of the entire population. Criminals might brag about their exploits to their posse but they are hardly likely to participate willingly in surveys.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Secondly, criminals can't be trusted so even if they did complete the forms, we probably shouldn't swallow their responses. Mind you, if the surveys weren't designed scientifically with extreme care over the precise questions, proper selection of the samples, rigorous statistical analysis, honest reporting etc., then all bets are off. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Thirdly, the police, governments/authorities, the news media, assorted commercial organizations, professions, industry bodies and pressure groups all have vested interests too, meaning that we probably shouldn't believe their surveys and assessments either, at least not uncritically*. Guess what, if an organization's income or power depends to some extent on the size of The Problem, they may, conceivably, allegedly, be tempted to slightly over-emphasize things, perhaps exaggerating, oh just a little and down-playing or ignoring inconvenient metrics and findings that don't quite align with their world view or objectives. [This one applies to me too as an infosec pro, but recognizing my inherent bias is not the same as counteracting it.]</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Fourthly, the metrics vary, for example in how they define or categorize crimes, what countries or areas they cover, and the measurement methods employed. Are US homicide numbers directly comparable with murders in, say, the UK? Are they even comparable, period-on-period, within any constituency? Would deliberately killing someone by running them over 'count' as a car crime, murder, accident, crime of passion, and/or what?</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Fifthly, the <i>effects </i>of crime are also hard to account for, especially if you appreciate that they extend beyond the immediate victims. Society as a whole suffers in all sorts of ways because of crime. These effects and the associated costs are widely distributed. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Sixthly, and lastly for now, crime is inherently scary, hence crime metrics are scary or eye-catching anyway. We risk losing our sense of perspective when considering 'facts' such as the <i>skyrocketing</i> rates of gun crime, home invasions, child abductions or whatever in relation to all the normal humdrum risks of everyday life, let alone all those scares about smoking, obesity, stress, heart disease and cancer. The emotional impact of crime metrics and the way they are portrayed in various media introduces yet more bias. [By the way, the same consideration applies to security metrics: perhaps we should explore that tangent another day.]</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">So, with all that and more in mind, what are we to make of cybercrime? How many cybercrimes are there? How many remain unidentified? To what extent can we trust our information sources? How do we even define, let alone measure, cybercrime? What is The Problem, and how big is it? And does it really matter anyway if the answer is <i>bound </i>to be scary?</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Well yes it does matter because all sorts of things are predicated on cybercrime statistics - strategies, policies (public, corporate and personal), risk assessments, investment and spending plans, budgets and so forth. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><a href="https://www.virusbulletin.com/virusbulletin/2016/02/vb2015-paper-sizing-cybercrime-incidents-and-accidents-hints-and-allegations/" rel="nofollow" target="_blank">The right answer might be: we don't know</a>. Good luck with all those predicates if that's your final answer! Phone a friend? 50/50?</span><br />
<span style="font-family: "verdana" , sans-serif;"><br /></span>
<span style="font-family: "verdana" , sans-serif;">* <b>Update Feb 20th: </b>according to </span><a href="http://www.hamiltonplacestrategies.com/news/cybercrime-costs-more-than-you-think#Paper" rel="nofollow" style="font-family: verdana, sans-serif;" target="_blank">Cybercrime costs more than you </a><span style="font-family: "verdana" , sans-serif;"><a href="http://www.hamiltonplacestrategies.com/news/cybercrime-costs-more-than-you-think#Paper" rel="nofollow" target="_blank">think</a>, "Cybercrime costs the global economy about $450 billion each year", a factoid used (for reasons that are not entirely obvious) to support a call for organizations to plan for incidents. Their sources are not clearly referenced but the paper appears to draw on a <a href="http://www.agcs.allianz.com/assets/PDFs/risk%20bulletins/CyberRiskGuide.pdf">glossy report by Allianz</a>, an insurance company with an obvious self-interest in pumping-up the threat level. The Allianz report in turn cited studies by the Ponemon Institute and by McAfee with the Center for Strategic and International Studies, three further organizations with axes to grind in this space</span><span style="font-family: "verdana" , sans-serif;">. To their credit, the 2014 </span><a href="http://www.mcafee.com/us/resources/reports/rp-economic-impact-cybercrime2.pdf" rel="nofollow" style="font-family: verdana, sans-serif;" target="_blank">McAfee/CSIS study</a><span style="font-family: "verdana" , sans-serif;"> openly</span><span style="font-family: "verdana" , sans-serif;"> acknowledged the poor quality of the available data - for instance stating: "... <i>we found two divergent estimates for the European
Union, one saying losses in the EU totaled only $16 billion, far
less than the aggregate for those EU countries where we could
find data, and another putting losses for the EU at close to a
trillion dollars, more than we could find for the entire world</i> ..." They also noted particular difficulties in estimating the costs of theft of intellectual property, while simultaneously claiming that IP theft is the most significant component of loss. Naturally, such carefully-worded caveats buried deep in the guts of the McAfee/CSIS study didn't quite make it through to the Allianz glossy or the sales leaflets that cite it. It's a neat example of how, once you unpick things, you discover that incomplete and unreliable information, coupled with rumours, intuition, guesswork, marketing hyperbole and weasel words, have morphed via factoids, soundbytes and headline horrors into 'fact'. Hardly a sound basis for strategic decision-making, or indeed for purchasing commercial goods and services. </span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-41264937181074795212016-02-10T10:58:00.000+13:002016-02-10T11:00:11.661+13:00Cause =/= Effect<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Animals like us are fantastic at spotting patterns in things - it's an inherent part of our biology, involving parts of our brains that are especially good at it. Unfortunately, while <i>some </i>patterns are significant, <i>many </i>are not, and our brains are not terribly good at differentiating between the two - in fact, we tend to overemphasize matches, believing them to be especially significant, meaningful and, in a sense, real.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">It could be argued that <i>both </i>pattern-recognition <i>and </i>overemphasis on matches are the result of natural selection over millenia, since in the wild, anything that helps us quickly identify and respond to possible attacks by predators, even if there are none, is likely to increase our survival, within reason anyway. Arguably, this is what makes wild animals 'alert', 'nervous' or 'jumpy'. It's a fail-safe mechanism. It's also the root of the fear we feel when we <i>think </i>we are in a dangerous situation, such as walking down a dark alleyway in an unfamiliar city at night. The sense of physical danger heightens our senses and primes our fight-or-flight instincts with a boost of adrenaline. Running away screaming from a harmless vagrant is safer than ignoring potential threats.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">However, what I've just done in that paragraph is invent a vaguely plausible scenario, outlined it briefly, and some of you now believe it to be true, based on nothing more than its apparent plausibility and my credibility (such as it is). The reason I mentioned running away screaming was to stimulate a visceral reaction in you: the strong emotions that situation invokes adds even more emphasis to the story. It 'makes sense'. In fact, there are <i>many </i>other plausible scenarios or reasons why pattern-recognition and overemphasis might or might not be linked to anything but having described a particular pattern, that is probably now locked into your brain and perhaps given special significance or meaning.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">To illustrate my point, look at pattern-recognition from the predator's perspective: predators need to recognize possible prey and respond ahead of competing predators ... but distinguishing edible prey from everything else (including other predators, animals with poisonous or otherwise dangerous defenses, and rocks) <i>is </i>a critical part of the predator's biology. Attacking anything and everything would be a fail-unsafe approach, the exact opposite of prey. In reality, there are very few 'pure' predators or prey: even prey animals need to eat, while apex predators at the very top of the food chain may have a fear of cannibalism or prey that successfully fights back, so the real world is far more complex that my simplistic description implies.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">OK, with that in mind, take a look at this graph:</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<a href="https://4.bp.blogspot.com/-GaVHlMHDEug/VrpYULDfgsI/AAAAAAAABjo/3m9mSs64oSk/s1600/2016-02-10%2B10_09_20-Spurious%2BCorrelations.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: "verdana" , sans-serif;"><img border="0" src="https://4.bp.blogspot.com/-GaVHlMHDEug/VrpYULDfgsI/AAAAAAAABjo/3m9mSs64oSk/s1600/2016-02-10%2B10_09_20-Spurious%2BCorrelations.jpg" /></span></a></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Sure looks like the red and black lines are related, doesn't it? They track each other, on the whole. Their patterns match quite closely over the 13 year period shown, implying that they are somehow linked. In that specific case, statistical analysis tells us that the two variables are indeed correlated with a probability of just under 79% where 100% represent total identicality (indistinguishable) and 0% represents total discrepancy (no relation whatsoever). 79% is a pretty high value, so it is entirely possible that the two variables are indeed linked. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">So, at this point we think we've found a link between <ahem> <i>the annual number of non-commercial space launches globally</i> and <i>the annual number of sociology doctorates awarded in the US</i> - for those are the numbers graphed! Hmmmm.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Yes, you might be able to come up with some vaguely credible reasoning to explain that apparent linkage, but be honest it would be a stretch of the imagination and would involve considerable effort to find, which you might be willing to do if you feel the pattern-match is somehow significant (!). Far more likely is that we've simply found a matching pattern, a sheer coincidence, a fluke. If we have enough data available and keep on searching, we can probably find other variables that appear to correlate with either of those two, including some with even higher coefficients of correlation ...</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">... which I guess is pretty much what someone has done - using automated statistical techniques to find correlations between published data. Have a browse through <a href="http://tylervigen.com/spurious-correlations" rel="nofollow" target="_blank">these spurious correlations</a> for some 29,999 other examples along these lines, and remember all this the next time you see a graph or a description that <i>appears </i>to indicate cause-and-effect linkages between anything. We humans desperately <i>want </i>to see matches. We find them almost <i>irresistable </i>and especially significant, almost magical, verging on real. Unfortunately, we are easily <i>deluded</i>.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">From that point, it is but a short hop to 'lies, damn lies, and statistics'. Anyone with an axe to grind, sufficient data and a basic grasp of statistics can probably find correlations between things that appear to bolster their claims, and a substantial proportion of their audience will be swayed by it, hijacked by their own biology. I rather suspect that civil servants, politicians and managers are pretty good at that.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">By the way, although I recognise the bias, I am far from immune to it. I <i>try </i>to hold back from claiming causal links purely on the basis of patterns in the numbers, and phrase things carefully to leave an element of doubt, but it's hard to fight against my own physiology.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Think on.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Gary.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">PS. Finding spurious matches in large data sets is an illustration of the <a href="http://betterexplained.com/articles/understanding-the-birthday-paradox/" rel="nofollow" target="_blank">birthday paradox</a>: there is a <i>surprisingly </i>high probability that two non-twin students in the average class were born on the same day. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">PPS The 79% correlation in the example above is only a fraction beneath the 'magical' 80% level. According to <a href="http://betterexplained.com/articles/understanding-the-pareto-principle-the-8020-rule/" rel="nofollow" target="_blank">Pareto's Principle</a> (I'm paraphrasing), 80% of stuff is caused by 20% of things. It's a rule-of-thumb that seems to apply in some cases, hence we subconsciouly believe it can be generalized, and before you know it, it's accepted as truth. The fact that 80% + 20% = 100% is somehow 'special' - it's another obvious but entirely spurious pattern.</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-33301054338514529822016-01-25T16:36:00.001+13:002016-02-20T11:19:54.979+13:00Metrics thought for the day<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Where relevant, using current business metrics (also) for information risk
and security purposes can be cost-effective if suitable raw data are already being gathered: the
additional analysis, reporting and use incur relatively little incremental cost, especially
if largely automated.</span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Corollary: when searching for metrics in any area of information risk and security, don't forget to check through existing business metrics alread in use for anything suitable, either as-is or with minor changes.</span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">It would be easier to identify such metrics if the organization maintained a <a href="http://securitymetametrics.blogspot.co.nz/2015/11/metrics-database.html" target="_blank">metrics inventory or database</a> ...</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-6999059822119109052015-11-20T16:44:00.004+13:002016-02-20T11:21:01.299+13:00Decision-led metrics<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Metrics in general are valuable because, in various ways, they support decisions. If they don't, they are at best just nice to know - 'coffee table metrics' I call them. If coffee table metrics didn't exist, we probably wouldn't miss them, and we'd have cut costs.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<a href="http://2.bp.blogspot.com/-mwn5A6MRPic/Vk6WtxLb5uI/AAAAAAAABg4/5so9x4zaHFA/s1600/Choices.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://2.bp.blogspot.com/-mwn5A6MRPic/Vk6WtxLb5uI/AAAAAAAABg4/5so9x4zaHFA/s1600/Choices.jpg" /></a><span style="font-family: "verdana" , sans-serif;">So, what decisions are being, or should be, or will need to be made, concerning information risk and security? If we figure that out, we'll have a pretty good clue about which metrics we do or don't want.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Here are a few ways to categorize decisions:</span></div>
<ul>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Decisions concerning strategic, tactical and operational matters, with the corresponding long, medium and short-term focus and relatively broad, middling or narrow scope;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Decisions about risk, governance, security, compliance ...;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Decisions about what to do, how to do it, who does it, when it is done ...;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Business decisions, technology decisions, people decisions, financial decisions ...;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Decisions about departments, functions, teams, systems, projects, organizations; </span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Decisions regarding strategies/approaches, policies, procedures, plans, frameworks, standards ...;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Decisions relating to threats, vulnerabilities and impacts - evaluating and responding to them;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Decisions made by senior, middle or junior managers, by staff, and perhaps by or relating to business partners, contractors and consultants, advisors, stakeholders, regulators, authorities, owners and other third parties;</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">Decisions about effectiveness, efficiency, suitability, maturity and, yes, decisions about metrics (!);</span></li>
<li style="text-align: justify;"><span style="font-family: "verdana" , sans-serif;">... [feel free to bring up others in the comments].</span></li>
</ul>
<br />
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Notice that the bullets are non-exclusive: a single metric might support strategic decisions around information risks in technology involving a commercial cloud service, for instance, putting it in several of those categories. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">If we systematically map out our current portfolio of security metrics (assuming we can actually identify them: do we even have an inventory or catalog of security metrics?) across all those categories, we'll probably notice two things. </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">First, for all sorts of reasons, we will probably find an <i>apparent </i>excess or surplus of metrics in some areas and a dearth or shortage elsewhere. That hints at <i>perhaps </i>identifying and developing additional metrics in some areas, and cutting down on duplicates or failing/coffee-table metrics where there seems to be too many which is itself a judgement call or a decision about metrics - and not as obvious as it may appear. Simplistically aiming for a "balance" of metrics across the categories is a naive approach</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Second, some metrics will pop up in multiple categories ... which is wonderful. We've just identified <b>key metrics</b>. They are more important than most since they evidently support multiple decisions. We clearly need to be <i>extra </i>careful with these metrics since data, analysis or reporting issues (such as errors and omissions, or unavailability, or deliberate manipulation) is likely to affect multiple decisions.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Overall, letting decisions and the associated demand for information determine the organization's choice of metrics makes a lot more sense than the opposite "measure everything in sight" data-supply-driven approach. What's the point in measuring stuff that nobody cares about? </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-24470875517609870842015-11-12T15:49:00.003+13:002016-02-20T11:27:59.787+13:00Metrics database<div style="text-align: justify;">
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-Jyiw-W1ia8g/VkP9rg0NGtI/AAAAAAAABgo/LoVwJiF8KBQ/s1600/Database.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="226" src="https://3.bp.blogspot.com/-Jyiw-W1ia8g/VkP9rg0NGtI/AAAAAAAABgo/LoVwJiF8KBQ/s400/Database.jpg" width="400" /></a></div>
<span style="font-family: "verdana" , sans-serif;">I wonder if any far-sighted organizations are using a database/systems approach to their metrics? Seems to me a logical approach given that there are lots of measurement data swilling around the average corporation (including but not only those relating to information risk, security, control, governance, compliance and privacy). Why not systematically import the data into a metrics database system for automated analysis and presentation purposes? Capture the data once, manage it responsibly, use it repeatedly, and milk the maximum value from it, right?</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">If you think that's a naive, impracticable or otherwise krazy approach, please put me straight. What am I missing? Why is it that I never seem to hear about metrics databases, other than generic metrics catalogs (which are of limited value IMNSHO) and Management Information Systems (which were all the rage in the 80s but strangely disappeared from sight in the 90s)?</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Conversely, if your organization has a metrics database system, how is it working out in practice? What can you share with us about the pros and cons?</span></div>
<div style="text-align: justify;">
<br /></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-23916169027878705242015-10-07T21:29:00.001+13:002016-02-20T11:29:12.126+13:00Security dashboard tips <div style="text-align: justify;">
<a href="http://4.bp.blogspot.com/-hrlyI266zlM/USPe4ptnm9I/AAAAAAAAAR0/JseYe2uuMgU/s1600/Dashboard.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://4.bp.blogspot.com/-hrlyI266zlM/USPe4ptnm9I/AAAAAAAAAR0/JseYe2uuMgU/s1600/Dashboard.jpg" /></a><span style="font-family: "verdana" , sans-serif;">Tripwire blog's <a href="http://www.tripwire.com/state-of-security/risk-based-security-for-executives/connecting-security-to-the-business/the-top-10-tips-for-building-an-effective-security-dashboard/" rel="nofollow" target="_blank">The Top 10 Tips for Building an Effective Security Dashboard</a> is an interesting collection of advice from several people. It's thought provoking, although I don't entirely agree with it.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Tip 2 'Sell success, not fear', mentions:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">"For example, in the event that they cannot find personnel who come equipped with the skills needed to improve progress, security personnel can use dashboards to demonstrate the impact that well trained individuals could have on finding and resolving issues and threats, as well as to subsequently leverage that insight for training and cultivating available skills."</span></blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Although somewhat manipulative, metrics can indeed provide data supporting or justifying proposed security improvements, assuming that, somehow, someone has already decided what needs to be done ... and suitable metrics can be useful for that purpose too.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The thrust of tip 4 'Use compelling visualizations' is that the dashboard needs to be glossy: I agree dashboards should be professionally crafted and reasonably well presented but I feel their true value and utility has far more to do with the information content than the look.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Tip 9 'Thoroughly vet the information before it is presented' is an odd one. The advice to be ready to explain outliers and anomalies makes sense, but the implication of someone vetting the data before it goes to the dashboard is that it will be both delayed and sanitized. Hmmm.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Well, take a look for yourself and see what you make of the ten tips.</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-34455470759121492152015-09-10T11:06:00.000+12:002015-09-10T11:06:21.944+12:00Metrics case study on Boeing<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-1vNZ0Nt41ds/VfC64b0CMLI/AAAAAAAABdA/wbYZ_NLVZX0/s1600/jet_engine%2Bmetric%2B600.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-1vNZ0Nt41ds/VfC64b0CMLI/AAAAAAAABdA/wbYZ_NLVZX0/s1600/jet_engine%2Bmetric%2B600.jpg" /></a></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The Security Executive Council has published an interesting case study concerning the <a href="https://www.securityexecutivecouncil.com/spotlight/?sid=29307&sc=NL509_spotBoeingCaseStudy#29307" rel="nofollow" target="_blank">review and selection of metrics relating to physical and information risks at Boeing</a>. [Access to the article is free but requires us to register our interest.]</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The case study mentions using SMART criteria and a few other factors to select metrics but doesn't go into details, unfortunately. Nevertheless, the analytical approach is worth reading and contemplating.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">If we were to conduct such an assignment for a client today, we would utilize a combination of tools and techniques across six distinct phases:</span></div>
<br />
<ol>
<li><div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Background information gathering concerning Boeing's business situation, information risks, and existing metrics, using standard analytical or audit methods, clarifying the as-is situation and building a picture of what needs to change, and why. This phase would typically culminate in a report and a presentation/discussion with management.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
</li>
<li><div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">GQM (Goal-Question-Metric) assessment eloquently described by Lance Hayden in <a href="http://www.securitymetametrics.com/html/hayden.html" target="_blank">IT Security Metrics</a>. This is a more structured and systematic version of the approach outlined in the case study. A workshop approach would be useful, probably several in fact to delve into various aspects with the relevant business people and experts. The output would be a matrix or tree-root diagram illustrating the goals, questions and metrics.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
</li>
<li><div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><b><span style="color: #660000;">PRAGMATIC </span></b>assessment and ranking of the metrics generated in phase 2 using the approach documented in <a href="http://tinyurl.com/PRAGMATICmetrix" rel="nofollow" target="_blank">our book</a>. The output would be a management report containing a prioritized list of metrics ranked according to their PRAGMATIC scores, leading to a further presentation/discussion with management and, hopefully, agreement on a shortlist of the most promising metrics, those actually worth pursuing. This and the previous phase would take a creative approach, thinking about what needs to be measured, why, how, when etc., using both GQM and </span><span style="font-family: Verdana, sans-serif;"><b><span style="color: #660000;">PRAGMATIC </span></b></span><span style="font-family: Verdana, sans-serif;">to firm-up the metrics that best fit the requirements </span><span style="font-family: Verdana, sans-serif;"> </span><span style="font-family: Verdana, sans-serif;">and focus groups to finalize the metrics (both existing metrics that are worth retaining possibly with some changes, and novel metrics being introduced).<br /></span></div>
</li>
<li><div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Planning and preparing for the implementation phase, perhaps including pilot studies.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
</li>
<li><div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Implementation: making the changes needed to collect, analyse, report and most of all <i>use</i> the metrics. This might well involve retiring or recasting some of the client's existing metrics that haven't earned their keep, in a way that teases out the last dregs of value from the data gathered previously.<br /></span></div>
</li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Ongoing metrics management and maintenance: using information from the GQM and <b><span style="color: #660000;">PRAGMATIC </span></b>steps to monitor and if appropriate refine or replace the metrics, ensuring for instance that they are proving valuable to the business (<i>i.e.</i> they should be cost-effective - one of the <b><span style="color: #660000;">PRAGMATIC </span></b>criteria conspicuously absent from SMART). </span></li>
</ol>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">In parallel with that sequence would be conventional project management activities - planning, resourcing & team building, motivation, tracking, reporting and assignment risk management.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><a href="http://www.securitymetametrics.com/html/contact_us.html" target="_blank">Get in touch to review and update your metrics</a>: we'd love to help!</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-31374502137518773372015-08-21T10:53:00.000+12:002015-08-21T10:53:08.087+12:00Lean security<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-twgZmQ38u6s/VdZZp_nEsII/AAAAAAAABbc/jRGT0e5ZSpI/s1600/Lean%2Bsecurity%2Bvert%2B1000.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://4.bp.blogspot.com/-twgZmQ38u6s/VdZZp_nEsII/AAAAAAAABbc/jRGT0e5ZSpI/s1600/Lean%2Bsecurity%2Bvert%2B1000.jpg" /></a></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><a href="https://en.wikipedia.org/wiki/Lean_manufacturing" rel="nofollow" target="_blank">Lean manufacturing</a> or </span><a href="https://en.wikipedia.org/wiki/Kaizen" rel="nofollow" style="font-family: Verdana, sans-serif;" target="_blank">kaizen</a><span style="font-family: Verdana, sans-serif;"> is a philosophy or framework comprising a variety of approaches designed to make manufacturing and production systems as efficient and effective as possible, approaches such as:</span></div>
<ul>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Design-for-life </b>- taking account of the practical realities of production, usage and maintenance when products are designed, rather than locking-in later nightmares through the thoughtless inclusion of elements or features that prove unmanageable;</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Just-in-time </b>delivery of parts to the production line at the quantity, quality, time and place they are needed (kanban), instead of being stockpiled in a warehouse or parts store, collecting dust, depreciating, adding inertia and costs if product changes are needed;</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Elimination of waste</b> (muda) - processes are changed to avoid the production of waste, or at the very least waste materials become useful/valuable products, while wasted time and effort is eliminated by making production processes slick with smooth, continuous, even flows at a sensible pace rather than jerky stop-starts;</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">An <i>obsessive, all-encompassing </i>and<i> continuous </i>focus on <b>quality assurance</b>, to the extent that if someone spots an issue anywhere on the production line, the <i>entire line </i>may be stopped in order to fix the root cause rather than simply pressing ahead in the hope that the quality test and repair function (a.k.a. Final Inspection or Quality Control) will bodge things into shape later ... hopefully without the customer noticing latent defects;</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Most of all, <b>innovation</b> - actively seeking creative ways to bypass/avoid roadblocks, make things better for all concerned, and deliver products that go above and beyond customer expectations, all without blowing the budget.</span></li>
</ul>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Service industries and processes/activities more generally can benefit from similar lean approaches ... so how might kaizen be applied to information risk management and security?</span></div>
<div style="text-align: justify;">
</div>
<ul>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Design-for-security</b> - products and processes should be designed from the outset to take due account of information security and privacy requirements throughout their life, implying that those requirements need to be elaborated-on, clarified/specified and understood by the designers;</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Just-in-case</b> - given that preventive security controls cannot be entirely relied-upon, detective and corrective controls are also necessary;</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Elimination of doubt </b>- identifying, characterizing and understanding the risks to information (even as they evolve and mutate) is key to ensuring that our risk treatments are necessary, appropriate and sufficient, hence high-quality, reliable, up-to-date information about information risk (including, of course, risk and security <b>metrics</b>) is itself an extremely valuable asset, worth investing in;</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Quality assurance </b>applies directly - information security serves the business needs of the organization, and should be driven by risks of concern to various stakeholders, not just 'because we say so';</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b>Innovation </b>also applies directly, as stated above. It takes creative effort to secure things cost-effectively, without unduly restricting or constraining activities to the extent that value is destroyed rather than secured.</span></li>
</ul>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-69914603353363322182015-08-04T09:24:00.003+12:002016-02-20T11:37:41.179+13:00Smoke-n-mirrors IBM style<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">I've just been reading the <a href="http://public.dhe.ibm.com/common/ssi/ecm/se/en/sew03073usen/SEW03073USEN.PDF" rel="nofollow" target="_blank">IBM 2015 Cyber Security Intelligence Index</a>, trying to figure out their 'materials and methods' <i>i.e.</i> basic parameters for the survey, such as population size and nature. All I can find are some obtuse references in the first paragraph:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">"IBM Managed Security Services continuously monitors billions of
events per year, as reported by more than 8,000 client devices in over
100 countries. This report is based on data IBM collected between
1 January 2014 and 31 December 2014 in the course of monitoring
client security devices as well as data derived from responding to and
performing analysis on cyber attack incidents. Because our client
profiles can differ significantly across industries and company size,
we have normalized the data for this report to describe an average
client organization as having between 1,000 and 5,000 employees, with
approximately 500 security devices deployed within its network."</span></blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Reading between the lines, it <i>appears</i> that this is a report gleaned primarily from 'more than 8,000 client [network security?] devices' belonging to an unknown number of organizations around the world who are customers of IBM Managed Security Services ... which <a href="http://us-cloud-new.ingrammicro.com/_layouts/CommerceServer/IM/CloudProductDetails.aspx?id=US02@@9500@@10@@79" rel="nofollow" target="_blank">IBM has described</a> as:</span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">"24/7/365 monitoring and management of security technologies you house in your environment. IBM provides a single management console and view of your entire security infrastructure, allowing you to mix and match by device type, vendor and service level to meet your individual business needs while drastically reducing your security costs, simplifying security management and accelerating your speed to protection."</span></blockquote>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">But, before you delve into the actual report, read that final sentence of the first paragraph again: they have 'normalized the data' (whatever that means) to an 'average client organization' with about 500 security devices ... so given the total of 8,000 devices, and on the assumption that 'average' means 'mean', it appears the survey covers </span><b style="font-family: Verdana, sans-serif;">just 16 organizations </b><span style="font-family: "verdana" , sans-serif;">whose network security devices are managed by IBM. </span><span style="font-family: "verdana" , sans-serif;">Oh boy oh boy. No wonder they are so reluctant to tell us about the analytical methods! </span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The data are from 2014, the report was published in July 2015. </span><span style="font-family: "verdana" , sans-serif;">Given the miniscule sample, I wonder why it took them 7 months to do the analysis and reporting? Crafting the words to gloss over the glaring flaws, perhaps?</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">The remainder of the report is pretty humdrum - some superficially interesting graphics and four 'case studies' (three of which - that's 75% or a 'vast majority', IBM - are not actual cases as such but fictional accounts based on the collective experiences of an unknown number of clients). </span><span style="font-family: "verdana" , sans-serif;">There's nothing particularly unusual or noteworthy in the report, despite the hyperbole (2014 was hardly "The year the Internet fell apart", IBM). The trends and other statistical information is worthless in scientific terms.</span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "verdana" , sans-serif;">Remember this cynical blog piece whenever you see the report quoted. Better still, read the report for yourself and make up your own mind.</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com1tag:blogger.com,1999:blog-7209707563895430269.post-49965320700831518592015-06-11T07:44:00.001+12:002015-06-11T07:44:53.880+12:00Culture metrics<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Over on Entrepreneur e-zine, serial company founder </span><a href="http://www.entrepreneur.com/article/246899" rel="nofollow" style="font-family: Verdana, sans-serif;" target="_blank">Greg Besner recommends the following ten metrics concerning organization's culture</a><span style="font-family: Verdana, sans-serif;">: </span></div>
<ol><a href="http://2.bp.blogspot.com/-t4CLmsqZoeI/VXiSU7d-RQI/AAAAAAAABVA/_048SoYPjDo/s1600/Measure.gif" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="207" src="http://2.bp.blogspot.com/-t4CLmsqZoeI/VXiSU7d-RQI/AAAAAAAABVA/_048SoYPjDo/s320/Measure.gif" width="320" /></a>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Communication</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Innovation</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Agility</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Wellness</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Environment</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Collaboration</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Support</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Performance focus</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Responsibility</span></li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">Mission and value alignment</span></li>
</ol>
<span style="font-family: Verdana, sans-serif;"><div style="text-align: justify;">
OK, but why did he pick <i>those </i>ten parameters to measure over all the others? What makes them <b>special</b>?</div>
</span><div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">In the article, Greg briefly explains his ten metrics in terms that make it clear why he thinks they are important. The trouble is, with just a moment's thought, I can easily come up with another ten, complete with my reasons for measuring them ... and I guess you too could come up with your self-justified list of ten culture metrics ... and so could anyone else with enough interest and expertise in this area ...</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">I guess right now you are puzzling over Greg's list, wondering about mine, and thinking about what else might be measured. Furthermore, I bet you are forming opinions about 'culture metrics' swimming around in your head, liking some, disliking others ... </span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">... and yet we haven't even attempted to reach agreement on a definition of "culture" at this point.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Ah, oh, yes.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">And furthermore, who said there had to be ten anyway? What's wrong with one, or three, or fifty seven?</span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">My point is that it's arbitrary. <i>My </i>choice of metrics - their number and their nature - almost certainly differs materially from <i>yours</i>. Both of us can justify our choices. Greg might feel compelled to defend his choice of ten. </span><span style="font-family: Verdana, sans-serif;">Given sufficient spare time and an ample supply of our favorite beverages, I'm sure we could have discussed cultural metrics for hours between us but somehow I doubt we would reach a consensus, for various reasons, not the least of which is that, in regard to metrics, <b>context matters</b>.</span><span style="font-family: Verdana, sans-serif;"> The cultural metrics that suit, say, a hi-tech start-up are likely to be different to those chosen by a government department, or an oil company, or a school. Any one of those organizations may choose different cultural metrics as it matures. Things that happen to be in vogue today may well change tomorrow, next week, next year or whatever (remember Peters & Waterman's "<a href="http://www.amazon.com/Search-Excellence-Americas-Best-Run-Companies/dp/0060548789/?_encoding=UTF8&camp=1789&creative=9325&keywords=in%20search%20of%20excellence&linkCode=ur2&pebp=1433965243462&perid=1DC28177633C497487D9&qid=1433965203&sr=8-1&tag=wwwnoticeborc-20&linkId=3UO25PL4H3GZFTAN" rel="nofollow" target="_blank">In search of Excellence</a>"? For a while, we obsessed about the characteristics that the book identified in excellent companies, but before long we realized there were many other important parameters too, and even Tom himself backtracked in his later books).</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-32529279555002334762015-05-25T10:18:00.000+12:002015-05-25T10:18:00.569+12:00Low = 1, Medium = 2, High = 97.1<div class="MsoPlainText" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div class="MsoPlainText" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Naïve risk analysis methods typically involve estimating the threats, vulnerabilities and impacts, categorizing them as low, medium and high and then converting these categories into numbers such as 1, 2 and 3 before performing simple arithmetic on them e.g. <i>risk = threat x vulnerability x impact</i>.</span></div>
<div class="MsoPlainText" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div class="MsoPlainText" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">This approach, while commonplace, is technically invalid, muddling up quite different types of numbers</span><span style="font-family: Verdana, sans-serif;">:</span></div>
<div class="MsoPlainText" style="text-align: justify;">
</div>
<ul>
<li><div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif; text-indent: -18pt;">Most of the time, numeric values such as 1, 2 and 3 are </span><b style="font-family: Verdana, sans-serif; text-indent: -18pt;">cardinal</b><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">
numbers indicating counts of the instances of something. The second value
(2) indicates twice the amount indicated by the first (1), while the third
value (3) indicates three times the first amount. Standard arithmetic <i>is
</i>applicable here.</span></div>
</li>
<li><div style="text-align: justify;">
<a href="http://3.bp.blogspot.com/-T49JrnGUa1o/VWJMH9E0H2I/AAAAAAAABUY/CBP0a5ZA_t4/s1600/Numeric%2Bkeypad%2Byellow%2B400.JPG" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="http://3.bp.blogspot.com/-T49JrnGUa1o/VWJMH9E0H2I/AAAAAAAABUY/CBP0a5ZA_t4/s320/Numeric%2Bkeypad%2Byellow%2B400.JPG" width="261" /></a><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">Alternatively, 1, 2 and 3 can indicate positions
within a defined set of values - such as 1</span><sup style="font-family: Verdana, sans-serif; text-indent: -18pt;">st</sup><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">, 2</span><sup style="font-family: Verdana, sans-serif; text-indent: -18pt;">nd</sup><span style="font-family: Verdana, sans-serif; text-indent: -18pt;"> and 3</span><sup style="font-family: Verdana, sans-serif; text-indent: -18pt;">rd</sup><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">
place in a running race. These </span><b style="font-family: Verdana, sans-serif; text-indent: -18pt;">ordinal</b><span style="font-family: Verdana, sans-serif; text-indent: -18pt;"> values tell us nothing
about how fast the winner was going, nor how much faster she was than the
runners-up: the winner might have led by a lap, or it could have been a
photo-finish. It would be wrong to claim that the 3</span><sup style="font-family: Verdana, sans-serif; text-indent: -18pt;">rd</sup><span style="font-family: Verdana, sans-serif; text-indent: -18pt;"> placed
entrant was “three times as slow as the 1</span><sup style="font-family: Verdana, sans-serif; text-indent: -18pt;">st</sup><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">” </span><i style="font-family: Verdana, sans-serif; text-indent: -18pt;">unless</i><span style="font-family: Verdana, sans-serif; text-indent: -18pt;"> you had
additional information about their speeds, measured using cardinal values and
units of measure: by themselves, their podium positions don’t tell you this. Some would say that being <span style="text-indent: -18pt;">1</span><sup style="text-indent: -18pt;">st</sup> is all that matters anyway: the rest are all losers. Standard arithmetic doesn't apply to ordinals such as threat values of 1, 2 or 3.</span></div>
</li>
<li style="text-align: justify;"><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">Alternatively, 1, 2 and 3 might simply have been
the numbers pinned on the runners’ shorts by the race organizers. It is
entirely possible that runner number 3 finished first, while runners 1 and 2
crossed the line together. The fourth entrant might have hurt her knee
and dropped out of the race before the start, leaving the fourth runner as
number 5! These are </span><b style="font-family: Verdana, sans-serif; text-indent: -18pt;">nominals</b><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">, labels that just happen to be digits
or strings of digits. Phone numbers and post codes are examples. Again, it makes no sense to multiply or subtract phone numbers or post
codes. They don’t indicate quantities like cardinal values do. If
you treat a phone number as if it were a cardinal value and divide it by 7, all
you achieved was a bit of mental exercise: the result is pointless. If
you ring the number 7 times, you still won’t get connected. Standard arithmetic makes no sense at all with nominals.</span></li>
</ul>
<div class="MsoPlainText" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">When we convert
ordinal values such as low, medium and high, or green, amber and red, risks
into numbers, they remain ordinal values, not cardinals – hence
standard arithmetic is inappropriate. If you convert back from ordinal numbers to
words, does it make any sense to try to multiply something by
"medium", or add "two reds"? “Two green risks”
(two 1’s) are not necessarily equivalent to “one amber risk” (a 2). In
fact, it could be argued that the risk scale is non-linear, hence “extreme”
risks are <i>materially</i> more worrisome than most mid-range risks, which are
of not much more concern than low risks. Luckily for us, extremes tend to
be quite rare! As ordinals, these risk numbers tell us only about the
relative positions of the risks in the set of values, not how close or distant
they are – but to be fair that is usually sufficient for prioritization and
focus. Personally, a green-amber-red spectrum tells me all I
need to know, with sufficient precision to make meaningful management decisions in
relation to treating the risks.</span></div>
<div class="MsoPlainText" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div class="MsoPlainText" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Financial risk analysis methods (such as SLE and ALE, or
DCF) attempt to predict and quantify both the probabilities and outcomes as
cardinal values, hence standard arithmetic applies … but don’t forget that
prediction is difficult, especially about the future (said Neils Bohr, shortly before
losing his shirt on the football pools). If you honestly believe your
hacking risk is precisely 4.83 times as serious as your malware risk, you are sadly deluded, placing undue reliance on the predicted numbers.</span></div>
<div class="MsoPlainText" style="text-align: justify;">
<o:p></o:p></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-4119825632329472542015-05-16T20:47:00.000+12:002015-05-16T20:47:47.448+12:00Metrics to govern and manage information security<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Section 9.1 of ISO/IEC 27001:2013 requires organizations to 'evaluate the information security performance and the effectiveness of the information security management system'. The standard doesn't specify precisely what is meant by 'information security performance' and '[information security?] effectiveness' but it gives some strong hints:</span></div>
<blockquote class="tr_bq">
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">"The organization shall determine:</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">a) what needs to be monitored and measured, including information security processes and controls;</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">b) the methods for monitoring, measurement, analysis and evaluation, as applicable, to ensure valid results;</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">c) when the monitoring and measuring shall be performed;</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">d) who shall monitor and measure;</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">e) when the results from monitoring and measurement shall be analysed and evaluated; and</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">f) who shall analyse and evaluate these results."</span></div>
</blockquote>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The standard specifies (much of) the measurement process without stating what to measure <i>i.e. </i>which metrics. No doubt the committee would argue that it is not possible to be specific about the metrics since each organization is different - and there's a lot of truth in that - but it's a shame they didn't explain how to select metrics or offer a few examples ... which is where our <a href="file:///D:/Users/Gary/PRAGMATIC/SecurityMetametrics%20website/SecurityMetametrics/Local%20Publish/63_NB_mgmt_briefing_on_infosec_governance_metrics.pdf" target="_blank">security awareness paper originally delivered in August 2008</a> picks up the pieces.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">We drew on the IT Governance Institute's advice on information security governance for inspiration, suggesting metrics corresponding to the four aspects identified in the ITGI paper (governance outcomes; knowledge & protection of information assets;</span><span style="font-family: Verdana, sans-serif;"> governance benefits; and process integration).</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">[The original hyperlink to the ITGI paper now gives a 404 page-not-found error, unfortunately. It was a good paper. Perhaps they moved or updated it?]</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-76887788449391267642015-05-07T21:27:00.001+12:002015-05-07T21:27:30.540+12:00Infosec & risk management metricsWe've just republished the next in the series of management-level security awareness papers on metrics. The latest one lays out a range of <a href="http://www.securitymetametrics.com/62_NB_mgmt_briefing_on_infosec_risk_mgmt_metrics.pdf" rel="nofollow" target="_blank">metrics for information security and risk management</a>.<br />
<br />
Leaving aside the conventional metrics that are typically used to manage <i>any </i>corporate function, the paper describes those that are peculiar to the management of information risk and information security, with an emphasis on business-focused metrics.<br />
<br />
I spent last week teaching a <a href="http://www.alctraining.com.au/course/cism-certified-information-security-manager/" rel="nofollow" target="_blank">CISM course</a> for ALC in Sydney. The business and risk focus is a unifying thread throughout CISM, from the governance and strategy angle through risk and security management to incident management. <br />
<br />
In contrast to courses covering the more technical/IT aspects of information security intended for mid- to low-level information security professionals with operational responsibilities, CISM is intended for Information Security Managers and Chief Information Security Officers with governance, strategic and management responsibilities. It promotes the value of elaborating on <b>business </b>objectives that are relevant to information risk and security management, and using those to drive the development and delivery of a coherent <b>business-aligned risk-driven </b>information security strategy. Metrics are of course integral to the CISM approach, particularly governance and management metrics similar to those in the awareness paper.Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-12935275343708664362015-04-24T10:54:00.003+12:002015-04-24T10:54:29.653+12:00Resilience as a business continuity mindset<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">An article written in conjunction with Dejan Kosutic has just been <a href="http://www.continuitycentral.com/index.php/news/resilience-news/175-feature1306" target="_blank">published at ContinuityCentral.com</a>. </span></div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">"<span style="background-color: white; color: #393939; line-height: 22.3999996185303px;">Most business continuity experts from an IT background are primarily, if not exclusively, concerned with establishing the ability to </span><strong style="background-color: white; color: #393939; line-height: 22.3999996185303px;">recover </strong><span style="background-color: white; color: #393939; line-height: 22.3999996185303px;">failed IT services </span><em style="background-color: white; color: #393939; line-height: 22.3999996185303px;">after </em><span style="background-color: white; color: #393939; line-height: 22.3999996185303px;">a serious incident or disaster. While disaster recovery is a necessary part of business continuity, this article promotes the strategic business value of </span><strong style="background-color: white; color: #393939; line-height: 22.3999996185303px;">resilience</strong><span style="background-color: white; color: #393939; line-height: 22.3999996185303px;">: a more proactive and holistic approach for preparing not only IT services, but also other business processes </span><em style="background-color: white; color: #393939; line-height: 22.3999996185303px;">before an incident</em><span style="background-color: white; color: #393939; line-height: 22.3999996185303px;"> in order that an organization will survive incidents that would otherwise have taken it down, and so keep the business operating in some form </span><em style="background-color: white; color: #393939; line-height: 22.3999996185303px;">during and following an incident."</em></span></blockquote>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">We explain how resilience differs from and complements more conventional approaches to business continui</span><span style="font-family: Verdana, sans-serif;">ty. It is </span><span style="font-family: Verdana, sans-serif;">a cultural issue with strategic implications and </span><span style="font-family: Verdana, sans-serif;">benefits for everyday routine business, not just in crisis or disaster situations. It has implications throughout the organization, including business activities/processes, systems, workers and relationships with third parties. It is an integral and essential part of risk management.</span></div>
<br />
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The article discusses resilience in the context of ISO 22301 and ISO27k, and includes a maturity model and metric to help organizations put the strategy into practice.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-ikY07ZaaTkg/VTl34wvUwRI/AAAAAAAABTQ/x8oIsm5qPsw/s1600/Maturity%2Bmodel%2Bcolour%2B500.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-ikY07ZaaTkg/VTl34wvUwRI/AAAAAAAABTQ/x8oIsm5qPsw/s1600/Maturity%2Bmodel%2Bcolour%2B500.gif" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Dejan and I share a passion for this topic that I hope comes across in our writing. Comments welcome!</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Regards,</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Gary (<a href="mailto:Gary@isect.com">Gary@isect.com</a>)</span></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-61675873002365249112015-04-21T21:35:00.001+12:002015-04-21T21:35:25.450+12:00Awareness paper on authentication and phishing metrics<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">We've just republished a management-level security awareness paper on metrics relating to <a href="http://www.securitymetametrics.com/61_NB_mgmt_briefing_on_phishing_metrics.pdf" target="_blank">user authentication and phishing</a>.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The introduction asks "How do we tell whether our authentication
controls are effective?" and "What does 'effective' even mean in this context?" - two decent questions that could be addressed through suitable metrics.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">Questions like these are central to the GQM (goal-question-metric) method (see <a href="http://www.securitymetametrics.com/html/hayden.html" target="_blank">IT Security Metrics</a> by Lance Hayden), and not just literally in terms of their position in the handy acronym. They link the organization's goals or objectives relating to information security, to the information security metrics that are worth measuring.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">In your particular circumstances, the <i>effectiveness</i> of authentication controls might or might not be of sufficient concern to warrant generating the associated metrics. Other aspects might take precedence, for example the amount invested in authentication controls, and the ongoing operating and maintenance costs of those controls. It's usually not too hard to think up a whole raft of aspects, parameters or concerns relating to the topic area, but focusing on the things that are likely to matter most to the organization (business priorities) is a good way to keep the list within reasonable bounds. Once you know what they are, the next step is to figure out the questions arising e.g. "Are we spending appropriately (neither too much nor too little) on authentication?"</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">From there, it's simply a matter of deciding what data would help address the questions, and those are your metrics! Job done! Errr, well, no, not quite: if you have several goals/areas of concern and numerous questions arising, each requiring multiple metrics to generate the answers, there is a distinct risk of being overwhelmed with possibilities. It is infeasible and in fact counterproductive to attempt to measure everything. Less is more! This is where <a href="http://www.securitymetametrics.com/html/sampler.html" target="_blank">the <b><span style="color: #660000;">PRAGMATIC </span></b>method</a> comes into play as a way to whittle down the long list to a shortlist of metrics showing the most promise. The GQM approach also suggests filtering out the metrics that don't address the questions very well, and trimming down on metrics addressing questions that are only marginally related to the organization's business goals. Both approaches have their merits.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<br /></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0tag:blogger.com,1999:blog-7209707563895430269.post-23325679083538023102015-04-10T10:08:00.000+12:002015-04-10T10:08:40.787+12:003 more metrics papers<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">We've just published another three documents on security metrics, written and first released five years ago as part of the management stream in the <a href="http://www.noticebored.com/" target="_blank">NoticeBored information security awareness service</a>.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The first paper was concerned with <a href="http://www.securitymetametrics.com/60_NB_management_briefing_on_integrity_metrics.pdf" target="_blank">measuring <b>integrity</b></a>. Despite being one of the three central pillars of information security, integrity is largely overshadowed by availability and, especially, confidentiality ... and yet, if you interpret 'integrity' liberally, it includes some extremely important information security issues. The 'completeness and correctness' angle is pretty obvious, while 'up to date-ness' and 'appropriateness' are less well appreciated. Add in the character and trustworthiness of people, and integrity takes on a rather different slant (Bradley Manning, Julian Assange and Edward Snowden springing instantly to mind as integrity failures). An 'honesty metric' is an innovative idea.</span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The integrity metrics paper also suggests measuring the integrity of the organization's security metrics program or system of measurements, on the basis that metrics ought to be accurate, complete, up-to-date and relevant. The metrics integrity issue is obvious when you think about it. Managing with poor quality information is less than ideal. However, in our experience, information security metrics are mostly taken at face value: we usually focus on what the numbers are telling us without even considering that they might perhaps be wrong, misleading, incomplete or inconsequential. Worse still, we get so distracted by the fancy "infographics" that the information content is almost irrelevant. That's hardly a scientific approach! We have raised this issue before in relation to treating published security surveys as gospel, blythely ignoring the fact that most are statistically dubious if not patently biased marketing copy. Remember this the next time you search the web for pie charts to illustrate your security investment proposals, or the next time someone tries to persuade you to loosen the purse strings! </span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">A short, humdrum paper on <a href="http://www.securitymetametrics.com/59_NB_management_briefing_on_IT_audit_metrics.pdf" target="_blank">IT audit metrics</a> suggests a few ways to measure the IT audit function, such as "IT audit program coverage" as well as conventional management metrics. </span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Verdana, sans-serif;">The third paper on <a href="http://www.securitymetametrics.com/58_NB_management_briefing_on_malware_metrics.pdf" target="_blank">malware metrics</a> was virtually the same as the version released <a href="http://www.securitymetametrics.com/46_NB_management_briefing_on_malware_metrics.pdf" target="_blank">a year earlier</a>. We made some changes the following year, partly due to the research and thinking that went into writing <a href="http://www.securitymetametrics.com/html/book.html" target="_blank">PRAGMATIC Security Metrics</a> ... but you'll have to wait just a bit longer for the 2009 paper.</span></div>
<div style="text-align: justify;">
<br /></div>
Garyhttp://www.blogger.com/profile/03271148849000325301noreply@blogger.com0