Posts - Integrating the
Model in the Real

1/13/09: Audit & Risk - Seeing the Forest from the Trees

9/4/08: Security ROI

6/28/08: Boise: A
Terrorist Target?

5/10/01: FFIEC Business
Continuity Planning

4/3/08: SOX 404 Audits

Return on Investment and Annualized Loss Expectancy

The search for a reliable method for determining a security ROI (the return on an investment in a security product or process) has been the Holly Grail of technology managers for decades. While security and technology managers within the company identify and develop proposals for implementing new security controls, the senior executives on the business side are the ones who have to be convinced to spend the money to implement those controls. Since most decisions on business investments rely heavily on the ROI or return on investment of the product or service (i.e. if the company spends x dollars on developing a product this year, how much will the product return in income over the next five years?), security professionals usually try to justify their requests to senior management for approval to purchase new controls by demonstrating how the control will either generate additional income or reduce future expenses. Typically the security ROI argument boils down to an analysis of the cost to purchase and implement a control (for example, the installation of a network intrusion monitoring tool) against an estimate of the future losses that would be incurred by the company due to undetected harmful activity on the network. Estimating future losses is, at best, an art form (see Bruce Schneier's Blog for an excellent discussion of the inherent problems with Security ROI) for at least two reasons:

  1. There is little or no historical data. If you apply for auto insurance or a credit card, the provider can refer to extensive amounts of historical data to determine the risk in providing you with the product. Nothing close to that amount of data exists in the operational risk area. This leads most practitioners to rely on personal experience and "gut feel" in estimating future losses.
  2. The playing field is constantly changing. If you look at how the tools thieves use to steal cars have evolved over the past ten years, you would likely see a limited number of changes. Compare that over the same period to how the criminals have adapted to changes in technology in order to remotely penetrate corporate and home computer systems. In the 1990's the primary threats were script kiddies blindly running malware and insecure hackers trying to make a name for themselves. If your web site was hacked, it typically was either taken off-line or the hacker posted their own front page to demonstrate their feat. Today organized crime has taken over and their tools change almost daily. Even if you can make a good guess as to future losses for an ROI estimate based on what you know of today's technology, tomorrow will likely prove you wrong.

Schneier provides a good example of the ROI rationalization process:

"The classic methodology is called annualized loss expectancy (ALE), and it's straightforward. Calculate the cost of a security incident in both tangibles like time and money, and intangibles like reputation and competitive advantage. Multiply that by the chance the incident will occur in a year. That tells you how much you should spend to mitigate the risk. So, for example, if your store has a 10 percent chance of getting robbed and the cost of being robbed is $10,000, then you should spend $1,000 a year on security. Spend more than that, and you're wasting money. Spend less than that, and you're also wasting money....

If you're doing an ALE analysis of a security camera at a convenience store, you need to know the crime rate in the store's neighborhood and maybe have some idea of how much cameras improve the odds of convincing criminals to rob another store instead. You need to know how much a robbery costs: in merchandise, in time and annoyance, in lost sales due to spooked patrons, in employee morale. You need to know how much not having the cameras costs in terms of employee morale; maybe you're having trouble hiring salespeople to work the night shift. With all that data, you can figure out if the cost of the camera is cheaper than the loss of revenue if you close the store at night -- assuming that the closed store won't get robbed as well. And then you can decide whether to install one."

The real world rarely supplies enough loss data to make the assessment of Financial Exposure foolproof. For this reason, we have added two further elements to the process to guide management in achieving a reasonable estimate.