Using FMEA to Measure Relative Risk

The fundamental role of intelligence is to support decision-making. And in the context of business, nearly every decision of importance has the potential to impact risk. This is why, in order to better enable effective decision-making, intelligence reporting must include a measure of relative risk.

So how do we, as decision-makers and intelligence practitioners, measure relative risk? We need to equip our arsenal with one simple tool: Failure Modes Effect Analysis (FMEA). FMEA is a framework designed to help users estimate the likelihood and impact of potential problems that could arise within a product or process. Reliability engineers originally developed FMEA in the late 1950s to study faults in military systems, but the tool has since gained traction among intelligence teams seeking to determine the relative risk posed by threats covered in their reporting.

Applying the FMEA framework in the context of an intelligence program requires teams to evaluate and score a given threat with respect to three characteristics:

1. Frequency: How often or likely is it that this threat will impact your organization?

2. Detection: If this threat does impact your organization, how likely are you to know?

3. Severity: If this threat does impact your organization, what impact will it likely have?

The following tables and criteria provide guidance on scoring for each characteristic:

Frequency: It may help to consider similar threats that have impacted your organization previously, as well as any emerging threats that have impacted similar organizations. For example, While an APT group seeking to compromise critical infrastructure is unlikely to target a retailer, that retailer may be a desirable target for cybercriminals interested in gift card fraud.

Detection: Your security operations center (SOC), security research team, or even red team may be able to help you more accurately assess detection scores, which typically depend on the capabilities and accuracy of your defenses and detection mechanisms, the expertise of your personnel, and the sophistication of the threat. For example, if a user’s device is infected with malware as a result of a phishing campaign, how quickly—if at all—would the SOC be able to identify it?

Severity: Your risk organization may be able to help you determine severity, and in some cases, they may even have dollar definitions for the scale itself. The threat on which you’re reporting could affect system availability, data integrity or confidentiality, physical security, or be measured in dollars. It’s important to remember that effective controls can greatly reduce severity. For example, if users have no ability to install applications, the severity of a particular phishing campaign would be greatly reduced.

After you’ve evaluated and populated the scores for each characteristic with respect to a given threat, multiply the three scores together (frequency score x detection score x severity score) to calculate the threat’s FMEA risk score, as follows:

Some iterations of this model will rank these scores in absolute terms because scores in certain ranges require certain actions, but doing so requires substantial experience with this tool. As such, you should consider these scores in relative terms.

In the above example, which threat is the least risky? While C has the lowest risk score and is therefore the least critical, there’s no significant difference between A and B given their scores. So in order to gain greater insight into the risks posed by A versus B, we need to take our analysis a step further.

By applying Business Risk Intelligence (BRI) and consulting with stakeholders to estimate mitigation costs for each threat, we can enhance our FMEA with dollar amounts, as follows:

Including mitigation costs in your FMEA not only elevates your intelligence reporting, it provides invaluable support and guidance to subsequent decisions regarding these threats. As you can see in the above example, while the company faces nearly the same risk from ATM skimmers as it does from insider recruitment, the drastic difference in mitigation costs suggests that the CISO will likely prioritize addressing ATM skimmers ahead of insider recruitment.

Before this FMEA, messaging to the CISO may have been: “Here are three intelligence reports on threats against our company.” But thanks to FMEA, the CISO can now make fully informed decisions about the actions and resource expenditures needed to mitigate the business risks posed by these threats.