Precision vs Accuracy When Predicting Security Cost
As I talk through the practical application of the security cost framework with our members, we are consistently running into a common stumbling...
After my last post on using Security Cost as a metric and the basis for a core security team/CISO objective, a few Mining and Metals ISAC members asked for a bit of a breakdown of what that metric looks like and how it can be used for a security team objective. While I did that today during our monthly ISAC call, I thought I'd share it here for the rest of you.
Security cost is an aggregate metric intended to capture enterprise-wide costs attributable to cybersecurity, regardless of whose cost/profit center they reside in or how they might appear on the income statement.
This one is easy, not going to talk too much about this other than to say you should be practicing 0-based budgeting, not just pulling forward the budget from last year and tweaking it.
it is critical when calculating security that all six forms of loss are captured. In addition, these costs should be captured for all incidents, from the minor to the material breach. All incidents should have costs in at least 2 of these categories:
Productivity Loss - The loss of time/production caused by the incident. This is from a user or business perspective.
Response Costs - Typically the security team's cost to respond, but should also include other internal and external response teams
Replacement Costs - Particularly relevant for cyber-physical incidents, if any equipment or other assets were damaged and needed to be repaired and replaced, this is where those costs are captured.
Competitive Advantage - Did you lose business as a result of the incident? Are you cutting prices to help keep and get new business? Capture those costs here.
Fines, Judgements, Penalties - Here, you capture regulatory, legal, and contractual impacts of the incident.
Reputation Damage - How does that get quantified? This one is hard, but it can include things like increased capital costs, share price impacts, and some will capture PR and IR costs here to counter the reputational impact. This can also include the impact on labour unions, regulators and other relationships.
Every security control we implement incurs a cost to some or all of the rest of the employees. Security Awareness training for 1 hour per quarter adds 4 hours of control costs per year. MFA adds control costs with every authentication... These are examples of small costs across a large population that must be estimated and aggregated. Larger costs might be time added to projects for security reviews, an extra dark fibre run for monitoring that is not paid for by the security budget.
This is what I call the 'Price of No'. Research shows that security concerns impact 40% of innovation projects... Projects cancelled, scope reduced(or increased) or material delay imposed. There are two costs here, first, for projects cancelled or scope reduced, there is the spend that is now wasted. For projects delayed or additional non-business requested scope added, there is an additional cost. For all cancelled and delayed projects, the largest cost is the missed opportunity. The business is now waiting longer, or will never see the benefit that the project was going to provide.
Security costs are a key aspect that traditional reporting metrics and objectives do not address: business integration and balance. If you have an objective that simply says "reduce security budget by 10%" you might make choices that reduce your budget (like say cut support staff) but result in longer waits for users, driving up control costs. Because those control costs are not within your budget, you are meeting your objectives at the expense of others in the organization. Similarly, if Control Maturity is an objective of yours, you might block an innovative project that does not fit into your control framework, costing other parts of the business.
Security cost solves these issues by forcing interaction with the business to gather the data, accountability for the positive and negative impacts of what you're doing, and providing a much broader basis to evaluate future security investments, as you'll be forced to consider the impact of that investment across all components of security cost, not just the impacts within your cost center.
The fringe benefit for CISOs is that it takes all the pressure off your security budget. In most cases, security spend is only 20-30% of the total. By working with the larger number (and for many, when they first calculate it, the number is shockingly high), your commitment in objectives to reduce that cost by as much as 20% will often result in a budget increase to allow you to go after the biggest contributors - and provide a solid business case for that increase.
Getting to the security cost is largely a change management and relationship exercise. I have not found an organization that cannot provide this data. However, 90% of the data the CISO, CIO or CTO does not have in their organization. It will require you to go out to the rest of the company, talk, survey, or otherwise discuss to get reliable data.
Don't have perfect data? Don't let perfection get in the way of improvement. Begin with calibrated estimates and refine their range until you get a narrow range. Your goal here is accuracy, not necessarily precision.
Once you have the number, communicate it. I would caution against doing a grand "unveiling" of this number at a board or Senior Management Team meeting. As I said, once you capture the number, it will likely be shockingly large. Discuss it with mid-senior leaders 1:1, in particular with operations, finance, and other teams that provided much of the input into the number. Be prepared to show your work, how you break down the big number into its smaller components that these leaders will recognize. Then, with their support, bring it to the relevant SMT member - that senior leader will be much more likely to accept the number if their own directs have already validated it and are in agreement of its accuracy (even if nobody likes the actual number)
Other great metrics I think CISOs should be tracking, reporting and building objectives against include:
Risks vs Risk Tolerance
Net Promoter Score
Team mental health
If there is interest, I'll happily dig into these ones as well. However, notice the metrics that are not there:
Security maturity, NIST score, BitSight score, Mean time to contain, resolve or other operational metrics. These metrics all are either operational (useful to the CISO and her managers to control the incident cost component of security cost) or I feel create tension and hurt cross functional relationships and just need to go away.
As I talk through the practical application of the security cost framework with our members, we are consistently running into a common stumbling...
For those of you who missed our first webinar of the year, I discussed a process for integrating IT and OT security —specifically, extending your IT...
As security practitioners and leaders, we must contribute to the professionalization of our field by searching out data and evidence-based solutions...