SMotW #40: ROI

Security Metric of the Week #40: ROI (Return On Investment)

ROI is a commonplace management accounting measure, a means of assessing the net worth (benefits less costs) of an investment, common as muck you might say.  But is is a good security metric?  

There are patently situations, in some organizations, in which ROI must be good since it often turns up in business cases, proposals or budget requests to help justify corporate projects or initiatives..  And yet some  prefer other financial metrics such as IRR (Internal Rate of Return).

When Acme Enterprises Inc. puts ROI through the PRAGMATIC sausage-machine, ROI turns out to be a fairly mediocre security metric: 

P	R	A	G	M	A	T	I	C	Score
65	72	25	25	88	50	44	60	90	58%

ROI is a Cost-effective metric (takes just an hour or two to calculate) that is presumably Meaningful to its intended audience i.e. management, but it doesn't score too well on the remaining criteria.  

It is not terribly Predictive of security outcomes.  'Throwing money at security' is no sure-fire way to become secure, while some penny-pinching organizations appear to survive with the bare minimum of security expenditure.  

The low Accuracy, Actionability and Genuinness ratings reflect concerns over its use as a decision-support tool.  ROI is normally used in isolation to justify individual projects that someone has already, in effect, chosen, rather than to compare a full suite of many possible investments, including various combinations and permutations (portfolio management) - meaning that a given security project may have a positive ROI, but various other security investments (not on the table) may have even better ROIs.  Even with the technical assistance of Finance Department to get the arithmetic right, ROI analyses for security investments necessarily require numerous assumptions, particularly in respect of the projected savings through mitigating security risks, reducing the probability and/or impact of security incidents.

Timeliness suffers because the metric is often only measured and reported prior to a project or initiative commencing, some months or years before it completes.  If Acme Enterprises was using ValIT (now subsumed within COBIT 5), it would be tracking actuals against projected costs and benefits, continually updating and refining the business case and ROI calculations.  But for the purposes of the book, we assumed Acme was just like most other organizations i.e. the ROI and business case is a one-off attempt to assess the  financials.