When can we stop assuming and start managing cyber-risk? It starts with data. When you get enough data on the likelihood of something bad happening, particularly at the security layer where risk is both at its greatest and its most unknown--your human layer, the place where 90% of data breaches begin—then you can start distinguishing between True Risk (capitalized intentionally) and assumed risk.
Revealing human risk begins with institutionalizing threat reporting. By reporting simulated phishing attacks, employees reveal their levels of cyber skill and weakness. This lets security teams make interventions at a granular level for people or units who struggle against phishing attacks. If you get enough people submitting enough threat reports over time, they're both building their cyber muscles and exposing True Risk.
Threat detection and visibility start with meaningful security training metrics. Quizzes don't cut it. You need adequate engagement and threat reporting rates for failure rates to mean anything. If you can get over half the enterprise regularly reporting simulated threats, you'll get a solid foundation of data that can ultimately shape the risk-based approach to cyber that today’s CISO needs and that Gartner strongly recommends.
The difference between True Risk and assumed risk is the difference between awareness and behavior change
Where traditional security awareness training is geared for checking a compliance box, a security behavior change program is designed around activity and engagement. Focusing training around hitting the threat report button yields the hard numbers that security teams need to stop breaches before they happen.
Meanwhile, a compliance-based awareness program is more spray-and-pray. Failure is punished, and not interacting with a phishing simulation is counted a success. Predictably, very little usable data emerges from such a program.
Reliable threat intelligence means the difference between True Risk and assumed risk.
And that’s crucial. True Risk paints the full picture of your organization’s preparedness for sophisticated attacks across business units. If one unit in one country is struggling with a particular type of attack, you can enhance training or take other precautions to mitigate that risk. True Risk means visibility into the phishing attacks that have evaded technical filters and infiltrated the system, but were caught by human intelligence.
True Risk is a science. Assumed risk is a data point. Decisions based on assumed risk are driven more by magical thinking than evidence.
Security behavior change reveals True Risk and drives human risk management
Engagement requires action. Security behavior change programs go beyond awareness by rewarding users for reporting a phishing simulation. Bad clicks might contribute to a higher phishing simulation failure rate, but they still provide important data and should not be punished. It's better that people are engaged and learning--and ultimately detecting real threats--than disengaged.
Learning doesn’t stop at training. When a user recognizes and reports a real threat, they're clarifying the the picture of True Risk; and lowering cyber-risk itself! A detected threat removes the danger from the ecosystem and gives security teams visibility into the threats that have evaded email filters. The ideal outcome of both phishing training and a real phishing attack is a threat report.
Resilience metrics
Measuring the True Risk of a phishing attack breach will produce higher resilience. First off, knowing the actual likelihood of your people clicking something they shouldn’t—or reporting something they should—will guide good business and security decisions. But strictly “measured risk” of a phishing attack breach can actually be dangerous.
Measured risk can look at failure rates out of context. Traditional awareness training tends to focus on failure, which leads to failure. Without adequate engagement, the failure rate is a mirage that can be based on:
- A poorly executed internal campaign
- Easy and ineffective training content.
- A vanity metric designed to make vendors and security teams look good by making phishing simulations easier and easier
Reporting risk to the board based on an empty metric is basically serving them junk food with empty calories; the sugar rush of saying, “Everything’s great!” will crash as soon as something bad actually happens and your team is held accountable for a suboptimal risk assessment.
What is the assumed risk of a phishing attack breach?
Employee phishing simulation pass / fail rates calculated in a vacuum. If only 100 employees in a 1000-strong workforce are participating in training, then the sample size renders their results—positive or negative—inadequate. Also, remember that a phishing tool can be designed to show improvement. What does that mean? Hard content that gets easier; or content that doesn’t effectively change, so the test takers can anticipate it and game the system.
Resilience ratio and True Risk
The resilience ratio provides a simple, handy dashboard metric for True Risk. Employee engagement in a security behavior change program should reach at least 50% of the organization, and ideally above 70%. These numbers will depend on the size of the organization.
From there, the CISO can calculate resilience and human risk with confidence. Just divide the engagement rate by the fail rate.
A score of 14 (e.g., 70% engagement / 5% clicked-a-bad-link rate) is excellent and worth striving for, while above 10-12 (60% engagement / 5-6% simulation fails) still provides your organization a competitive advantage. The Platonic ideal of 20-40 (80% engagement / 2-4% fail rate) is rare, but possible. Several Hoxhunt users have reached scores all the way into the mid-30s.
Mind you; the engagement must be honest. It can’t mean someone took one test, passed, and then was removed from testing but remains counted as a participant. Simulations must be challenging and touch the organization's upper echelons, just as sophisticated spear-phishing and whaling attacks do. Engagement cannot be faked or taken for granted. People must constantly be stressed with true-to-life threat simulations that evolve along with the threat landscape. Only then do pass/fail rates of threat simulations provide meaningful data for the infosec team to report to executive leadership with confidence.
The resilience funnel
The graphs below depict how True Risk looks in practice. We call this the resilience funnel. As you'll see, companies that use a traditional SAT solution have very high miss rates and very low success rates. Typically, simulated (and real) phishing threat reports are virtually nonexistent when they first graduate from standard SAT tools to a security behavior change program.
Measuring the True Risk effectively unlocks new levels of resilience via a risk-based approach. Just when you thought it was safe to get back in the water... it turns out it really is safe, because you can see risk clearly. Look at how the security behavior change program inverted the blue with the red, while flatlining the failure rate. In practice, this graph below indicates that even after dozens of phishing simulations over periods of years, people continued staying engaged and providing security teams with visibility into True Risk.
In an analysis of 1.6 million Hoxhunt users from over 100 countries responding to over 27 million phishing simulations, the picture of true risk was revealed: it's a funnel of human threat detection activity that expands over time, as missed and failed phishing simulations dwindle. The ideal outcome of a real or simulated phishing attack is a threat report. High rates of real threat detection are associated with reported simulated threats. Conversely, low rates of threat detection are associated with low engagement with simulated phishing attacks—what Hoxhunt calls the “Miss rate.” The failure rate illuminates the organization's risk posture only once engagement rate is satisfactorily high.
Security Behavior Change programs give a clear picture of True Risk.
The cybersecurity awareness community is fixated on failure when it should be focused on success. In the inaugural Behavioral Cybersecurity Statistics report, Hoxhunt analyzed the results of 1.4 million users’ responses to over 24 million phishing simulations. There were three possible outcomes:
Not interacting with a phishing simulation = Miss
Successfully reporting a phishing simulation as a threat = Success
Mistakenly clicking on a simulated phishing link = Failure
Guess which of these outcomes was most linked to breaches and cyber-risk? You are incorrect if you answered “failure,” as the industry typically would. It's a ‘Miss.” High miss rates—which translate to low training participation—correlate to a higher risk of a breach and a far lower likelihood of threat detection. Remember, the ideal outcome of a phishing attack is a threat report. When people report threats, they remove the danger from the ecosystem and alert the SOC team to activate response.
Why is non-participation and assumed risk a problem?
In addition to overlooking engagement, several design flaws must be improved to improve participation rates. Traditional punitive training programs are:
- Irrelevant and uninteresting: dry, stale, cookie-cutter content is force-fed rather than served in a personalized recipe, which invites active and ongoing participation
- Failure is punished via Death by a Thousand Bad Simulations: cybersecurity is thus associated with misery
- Compliance-based means training is made for everyone except for the people taking it: it’s a test-first philosophy, not people-first
- Premised on pass/fail rates, regardless of their statistical significance: If only 10% of your workforce is participating in these tests, results are non-representative
Engagement is the bedrock of effective training and learning. It’s a pillar of meaningful risk and resilience data at the people layer. Not only does org-wide engagement lower unknown risk, but the act of engagement—reporting threats, both real and simulated, auto-enhances protect-detect-respond capabilities:
- Behavior change, rooted in neuroscience, replaces an undesired activity with a healthy one to build good habits, from smoking (replaced with gum chewing or exercise) to unsafe cybersecurity practices (see something suspicious? Hit the report button!): Protect
- Strengthens the organization’s defensive perimeter at the people layer to lower bad clicks: Protect
- Serves as a distributed phishing defense tactic. Imagine transforming the human layer from an expansive threat surface into 20,000 active threat detection sensors: Detect
- Integrating threat detection into the security stack lets people and security join forces to enhance and accelerate threat response, so incidents can be prevented or contained before spreading
Unknown risk is dangerous waters
Verizon’s Data Breach Investigation Report indicates that traditional phishing awareness training obscures an organization’s true risk of a breach.“Additionally, real phishing may be even more compelling than simulations,” stated the report. “In a sample of 1,148 people who received real and simulated phishes, none of them clicked the simulated phish, but 2.5% clicked the real phishing email.” Click rates are typically far worse than that, even: between 7.5% - 49% depending on the industry and organization, according to a major 2018 study of phishing click rates across 6 US hospitals, published in JAMA by Gordon et al. The scientists reported 95 simulated phishing campaigns comprising 2 ,971, 945 emails produced a median click rate of 16.7 % across the 6 hospitals. They said that the median institutional click rates varied from 7.4% to 30.7%; 1 in 7 phishing simulations were clicked.
But here’s the part that should make you lean forward in your chair and smile: the study authors noted that “increasing campaigns were associated with decreased odds of clicking a phishing email.” Engagement works. Science says so. Institutional knowledge agrees. However, the quality of that engagement is crucial. In addition to failing to achieve adequate, much less optimal, engagement rates, the DBIR report further derided traditional training programs: “Verizon Media believes the simulations and training offered by most security education teams don't mimic real life situations, don't parallel the behaviors that lead to breaches, and are not measured against real attacks the organization receives. This is why it is important to progress from the traditional security awareness model to that o using behavioral science to change the habits (emphasis ours) that lead to attack path breaking actions.” —DBIR 2021A big part of the CISO’s job is to raise awareness. And not just of his or her employees. Executive management, too. Just as bad phishing training will likely not move the awareness needle and kill cybersecurity culture, poorly measured risk will introduce an element of voodoo into the risk analysis delivered to the board.