Human risk must be measured before it can managed. But not all measurements are alike. There is True Risk--calculated from user performance with an organizational engagement level above 50% --and then there is unknown risk, which will persist if performance is measured solely on failure rate without context of engagement or threat detection skill.
When can we stop assuming and start managing cyber risk? It starts with data. When you get enough data on the likelihood of something bad happening, particularly at the security layer where risk is both at its greatest and its most unknown--your human layer, the place where 90% of data breaches begin--then you can start distinguishing between True Risk (capitalized intentionally) and assumed risk.
Revealing human risk begins with institutionalizing threat reporting. By reporting simulated phishing attacks, employees reveal their levels of cyber skill and weakness. This lets security teams make interventions at a granular level for people or units who struggle against phishing attacks. If you get enough people submitting enough threat reports over time, they are both building their cyber muscles and exposing True Risk.
Threat detection and visibility start with meaningful security training metrics. Quizzes don't cut it. You need adequate engagement and threat reporting rates for failure rates to mean anything. If you can get over half the enterprise regularly reporting simulated threats, you'll get a solid foundation of data that can ultimately shape the risk-based approach to cyber that today's CISO needs and that Gartner strongly recommends.
Where traditional security awareness training is geared for checking a compliance box, a security behavior change program is designed around activity and engagement. Focusing training around hitting the threat report button yields the hard numbers that security teams need to stop breaches before they happen.
Meanwhile, a compliance-based awareness program is more spray-and-pray. Failure is punished, and not interacting with a phishing simulation is counted a success. Predictably, very little usable data emerges from such a program.
Reliable threat intelligence means the difference between True Risk and assumed risk.
And that’s crucial. True Risk paints the full picture of your organization’s preparedness for sophisticated attacks across business units. If one unit in one country is struggling with a particular type of attack, you can enhance training or take other precautions to mitigate that risk. True Risk means visibility into the phishing attacks that have evaded technical filters and infiltrated the system, but were caught by human intelligence.
True Risk is a science. Assumed risk is a data point. Decisions based on assumed risk are driven more by magical thinking than evidence.
Engagement requires action. Security behavior change programs go beyond awareness by rewarding users for reporting a phishing simulation. Bad clicks might contribute to a higher phishing simulation failure rate, but they still provide important data and should not be punished. It's better that people are engaged and learning--and ultimately detecting real threats--than disengaged.
Learning doesn't stop at training. When a user recognizes and reports a real threat, they are clarifying the the picture of True Risk; and lowering cyber risk itself! A detected threat removes the danger from the ecosystem and gives security teams visibility into the threats that have evaded email filters. The ideal outcome of both phishing training and a real phishing attack is a threat report.
Measuring true risk of a phishing attack breach will produce higher resilience. First off, knowing the actual likelihood of your people clicking something they shouldn’t—or reporting something they should--will guide good business and security decisions. But strictly “measured risk” of a phishing attack breach can actually be dangerous.
Measured risk can look at failure rates out of context. Traditional awareness training tends to focus on failure, which leads to failure. Without adequate engagement, the failure rate is a mirage that can be based on:
Reporting risk to the board based on an empty metric is basically serving them junk food with empty calories; the sugar rush of saying, “Everything’s great!” will crash as soon as something bad actually happens and your team is held accountable for a suboptimal risk assessment.
Employee phishing simulation pass / fail rates calculated in a vacuum. If only 100 employees in a 1000-strong workforce are participating in training, then the sample size renders their results—positive or negative—inadequate. Also, remember that a phishing tool can be designed to show improvement. What does that mean? Hard content that gets easier; or content that doesn’t effectively change, so the test takers can anticipate it and game the system.
The resilience ratio provides a simple, handy dashboard metric for True Risk. Employee engagement in a security behavior change program should be at a level of at least 50% of the organization, and ideally above 70%. These numbers will depend on the size of the organization.
From there the CISO can calculate resilience and human risk with confidence. Just divide engagement rate by fail rate.
A score of 14 (e.g. 70% engagement / 5% clicked-a-bad-link rate) is excellent and worth striving for, while above 10-12 (60% engagement / 5-6% simulation fails) still provides your organization competitive advantage. The Platonic ideal of 20-40 (80% engagement / 2-4% fail rate) is rare, but possible. Several Hoxhunt users have reached scores all the way into the mid-30s.
Mind you, the engagement must be real. It can’t mean someone took one test, passed, and then was removed from testing but remains counted as a participant. Simulations must be challenging, and touch the upper echelons of the organization, just as sophisticated spear phishing and whaling attacks do. Engagement cannot be faked or taken for granted. People need to be constantly stressed with true-to-life threat simulations that evolve along with the threat landscape. Only then do pass / fail rates of threat simulations provide meaningful data for the infosec team to report to executive leadership with confidence.
Measuring the True Risk effectively unlocks new levels of resilience via a risk-based approach. Just when you thought it was safe to get back in the water... it turns out it really is safe, because you can see risk clearly. Look at how the a security behavior change program inverted the blue with the red, while flatlining the failure rate. In practice, this graph below indicates that even after dozens of phishing simulations over periods of years, people continued staying engaged and providing security teams with visibility into True Risk.
The cybersecurity awareness community is fixated on failure when it should be focused on success. In the inaugural Behavioral Cybersecurity Statistics report, Hoxhunt analyzed the results of 1.4 million users' responses to over 24 million phishing simulations. There were three possible outcomes:
Not interacting with a phishing simulation = Miss
Sucessfully reporting a phishing simulation as a threat = Success
Mistakenly clicking on a simulated phishing link = Failure
Guess which of these outcomes was most linked to breaches and cyber risk? If you answered "failure," as the industry typically would, you are incorrect. It's a 'Miss." High miss rates--which translate to low training participation--correlate to higher risk of a breach and far lower likelihood of threat detection. Remeber, the ideal outcome of a phishing attack is a threat report. When people report threats, they remove the danger from the ecosystem and alert the SOC team to activate response.
Here is how that picture of True Risk looks in practice. The first image shows the We call this the resilience funnel. As you'll see, companies who use a traditional awareness training solution have very high miss rates and very low success rates. Typically, simulated (and real) phishing threat reports are virtually non-existent when they graduate from standard awareness training to a security behavior change program.
https://www.hoxhunt.com/behavioral-cybersecurity-ebook
In addition to overlooking engagementseveral design flaws drive poor participation rates. Traditional punitive training programs are:
Engagement is the bedrock of effective training and learning. It’s a pillar of meaningful risk and resilience data at the people layer. Not only does org-wide engagement lower unknown risk, but the act of engagement—reporting threats, both real and simulated, auto-enhances protect-detect-respond capabilities:
As indicated by Verizon’s Data Breach Investigation Report, traditional phishing awareness training obscures an organization’s true risk of a breach.“Additionally, real phishing may be even more compelling than simulations,” stated the report. “In a sample of 1,148 people who received real and simulated phishes, none of them clicked the simulated phish, but 2.5% clicked the real phishing email.” Click rates are typically far worse than that, even; between 7.5% – 49% depending on the industry and organization, according to a major 2018 study of phishing click rates across 6 US hospitals, published in JAMA by Gordon et al. The scientists reported 95 simulated phishing campaigns comprising 2 ,971, 945 emails produced an overall median click rate of 16.7 % across the 6 hospitals. The median institutional click rates per varied from 7.4% to 30.7%; so, 1 in 7 phishing simulations were clicked, they said.
But here’s the part that should make you lean forward in your chair and smile: the study authors noted that “increasing campaigns were associated with decreased odds of clicking a phishing email.” Engagement works. Science says so. Institutional knowledge agrees. However, the quality of that engagement is crucial. In addition to failing to achieve adequate, much less optimal, engagement rates, the DBIR report further derided traditional training programs:“Verizon Media believes the simulations and training offered by most security education teams do not mimic real life situations, do not parallel the behaviors that lead to breaches, and are not measured against real attacks the organization receives. This is why it is important to progress from the traditional security awareness model to that of using behavioral science to change the habits (emphasis ours) that lead to attack path breaking actions.” —DBIR 2021A big part of the CISO’s job is to raise awareness. And not just of his or her employees. Executive management, too. Just as a bad phishing training will likely not move the awareness needle and kill cybersecurity culture, poorly measured risk will introduce an element of voodoo into the risk analysis delivered to the board.
