Security Metrics: Replacing Fear, Uncertainty, and Doubt

Upon completion of this book, I began to muse: what percentage of security professionals have given any thought to security metrics? For those that have actually considered the topic, with what level of frequency do they entertain thoughts of security metrics? Yearly? Monthly? Daily? Gee, I think to myself, I'd like to see a time series analysis exhibit of that...Based on the fact that I sit here torturing myself with these thoughts, I contend that Security Metrics has already influenced my approach toward security management. Indeed, Jaquith has done an excellent job of exposing an area that is critical to effective security management, but to which many security practitioners (myself included) have previously paid lip service. Security Metrics offers valuable insight to organizations seeking to provide a greater level of intelligence and meaning around their security program(s).In addition to how well the ideas of the book resonated with my own professional and academic background, the choice to give a 5 star rating was based on its organization, readability, entertaining quips, and the fact that many of the alternative publications in the realm of security metrics are triple or more the cost of this one. Though I've not yet read or reviewed other similar works, the bar has been set high.

SECURITY METRICS is one of the only books you can find dealing with Info System Security (ISS) metrics. Author is a consultant and offers best practices on how to present metrics (aesthetics), and advises you on what tangible metrics will give you the most bang for the buck. Later in the book, Jaquith takes you up to the next level by adapting the Balanced Scorecard to the ISS world. Again, author walks you through specifics on metrics that would be reflective of the four different perspectives [Financial, Customer, Internal, and Learning&Growth]...a big help for anyone who has wrestled with Kaplan & Norton's "Balanced Scorecard" book.

See the Full Review at my blog site: Terebrate.This book is a must-read for all cyber security professionals. It is not a part of the canon because it attacks a sacred cow of the industry—Annualized Loss Expectancy (ALE) as a means to justify your security budget—and the community has yet to fully embrace the idea that ALE might not be a good idea in all cases. But you should seriously consider this notion and this book is your gateway to do so. Consider it a Canon-Candidate. Jaquith describes why capturing and analyzing security metrics is a good and powerful thing and how you can use that intelligence to better understand the porous nature of your networks. It will help you unshackle yourself from the chains of probabilistic risk assessments. It will turn you away from the dark side and toward a more meaningful process to assess your enterprise’s security. You should have read this by now.

A must read if you manage information security at your organization, Jaquith is an insightful manager and excellent writer sharing his thoughts and examples.

From the examples, you can tell this book is a bit dated. Other than that, it is still very crisp and fresh. The message that it advocates is, in my experience, still very welcome in this field (and beyond). Recommended!

Interesting and useful info.

What a book. Seriously, I laughed, I cried. I shouted in frustration, only to be placated on the next page. I got a better understanding of what Andy has been banging on about with Security Metrics. And it helps me do my job better.

The goal of security metrics is to replace fear, uncertainty, and doubt (FUD) with a more formalized and meaningful system of measurement. The FUD factor is the very foundation upon which much of information security is built, and the outcome is decades of meaningless statistics and racks of snake oil products. Let's hope that Andrew Jaquith succeeds, but in doing so, he is getting in the way of many security hardware and software vendors whose revenue streams are built on FUD.One could write a book on how FUD sells security products. One of the most memorable incidents was in 1992 when John McAfee created widespread panic about the impending Michelangelo virus. The media was all over him as he was selling solutions for the five million PCs worldwide he said would be affected. The end result is that the Michelangelo virus was a non-event. Nonetheless, it was far from the last time that FUD was used to sell security.The allure of FUD is that companies can spend huge amounts of money fighting nebulous digital adversaries and feel good about it. They can then put all of that fancy hardware in dedicated racks in their data center, impressing the auditors with the flashing lights giving off an aroma of security and compliance.And that is the chaos that security metrics comes to solve. Security metrics, if done right, can help transform a company from a nebulous perspective on security to an effective one based on formal security risk metrics.Security Metrics is a fabulous book that should be in the hands of every security professional. The book demonstrates that companies must establish metrics based on their unique requirements, as opposed to simply basing their requirements on imprecise industry polls, best-practices and other ill-defined methods.So why don't companies do that in the first place? If security metrics can provide even a quarter of the benefits that Jaquith states, companies should run to implement them. Real security metrics require an organization to open up their security hood and dig deep into the engine that runs their security infrastructure. It necessitates understanding the internal requirements, unique organizational risks, myriad strengths and weaknesses, and much more. Very few companies are willing to dedicate the time and resources for that, and would rather build their security infrastructure on thick layers of FUD. History has shown that the security appliance of the month almost always beats a formal risk and needs assessment.Chapter 1 lays out the problem with approaches that most companies take to risk management. The main problem is that traditional risk management is far too dependant on identification and fixing, as opposed to quantification and triage based on value. Quantifying and valuing risk is much more difficult than simply identifying, since the software tools used do not have an organization context or knowledge of the specific business domain.Chapter 2 sets out the foundation of security metrics. The goal of these metrics are to provide a framework in which organizations can quantify the likelihood of danger, estimate the extent of possible damage, understand the performance of their security organizations and weigh the costs of security safeguards against their expected effectiveness.The time has come for security metrics since information security is one of the few management disciplines that have yet to submit itself to serious analytical scrutiny. The various chapters provide many different metrics that can be immediately used in most organizations to address that.The author defines various criteria for what makes a good metric. One of his pet peeves is the use of the traffic light as a metaphor for compliance. Jaquith feels that traffic lights are not metrics at all, since they don't contain a unit of measure or are a numerical scale. He suggests using traffic lights colors sparingly, and only to supplement numerical data or draw attention to outliers. He astutely notes that if your data contains more precision than three simple gradations, why dilute their value by obscuring them with a traffic light.The chapter concludes on what makes a bad metric, defined as any metric that relies too much on the judgment of a person. These metrics can't be relied on since the results can't be guaranteed to be the same from person to person. Also, security frameworks such as ISO-17799 should not be used for metrics. The book also tackles the sacred cow of risk management, namely ALE (annualized loss expectancy), and how it is significantly misused and misunderstood in the industry.The book states that in developing metrics, there must be formal collaboration between the business units and the security staff. This collaboration serves to increase awareness and acceptance of security. In addition, it ensures that security requirements are incorporated into the lifecycle early on. This is needed as business units generally have no clue as to what the needed security requirements are.Chapter 5 is a short course on analysis techniques and statistics. The author quotes George Colony who stated that "any idiot can tell you what something is. It is much harder to say what that thing means". With that, the book details a number of techniques for analyzing security data (average, median, time series, etc.) and how each one should be used.Chapter 6 is about visualization and notes that most information security professionals have no real idea how to show security, both literally and figuratively. Part of the problem is that security is proliferated with esoteric terminology and concepts, and the lack of understanding risk management amongst the masses. Part of the reason for this difficulty in sharing the security message with management is that many security practitioners lack simple metaphors for communicating priorities. This is compounded by the fact that the message is often focused exclusively on technical security issues, as opposed to the underlying business issues, which is was management is concerned with. The chapter is invaluable as it weans one off the malevolent pie chart and traffic light PowerPoint presentation.Marcus Ranum notes that people seem to want to treat computer security like its rocket science or black magic. In fact, computer security is nothing but attention to detail and good design. FUD is all about emphasizing the black magic aspect of hackers and other rogue threats. Metrics are all about the attention to detail that FUD lives to obfuscate.Security Metrics: Replacing Fear, Uncertainty, and Doubt is one of the more important security books of the last few years. Jaquith turns much of the common security wisdom on its head, and the world will be a better place for it. Security metrics are a necessity whose time has come and this invaluable book shows how it can be done.