Biggest hidden risk in cybersec: missed phishing simulations

Cybersecurity awareness training traditionally fixates on failure. But in the world of cybersecurity, it’s the unknowns that will get you. After analyzing the responses of 1.6 million users to 24.7 million phishing simulations, we at Hoxhunt discovered something fascinating. The Miss Rate—the portion of phishing simulations that users neither report nor fail—represents both the biggest unknown and the biggest risk of a phishing breach. All those missed phishing simulations? They'll come back to bite you.

Post hero image

Table of contents

Take a look at the graph below. It is unique. This image tells the story of how 1.6 million users' practice with 24.7 million phishing simulations paid off in terms of:

1. Cybersecurity skill: people get better at recognizing and reporting phishing attacks

2. Lowering risk of a phishing breach: you know, the ones that are costing companies on average around $14 million/breach

3. Revealing true risk

But look closer. What trend lines have the most activity; and proportionality? And why is fail rate more or less flat-lined beneath that big inversion betwen miss rate and success rate?

This is where things get really interesting.

Mind your misses: a miss today is a phish tomorrow

This graph contains a million-dollar  insight into the great unknown of employee risk: the "missed" phishing simulations. Go ahead and look at the fail rate. See how it dips, and then levels out? That is certainly good. It means fewer people are failing fewer phishing simulations.

The X-axis is the number of phishing simulations over 2 years. The Y-Axis is the out-of-100 proportionality of user responses, by Fail, Miss, or Success.
The X-axis is the number of phishing simulations over 2 years. The Y-Axis is the out-of-100 proportionality of user responses, by Fail, Miss, or Success.

But look at how those two lines above the fail rate--"miss" and "success"--beautifully diverge. This activity signifies sustainable, ongoing skills acquisition and the revelation of true risk and resilience. The misses (those phishing simulations that go neglected by the user) transform directly into success (correctly reported phishing simulations). With Hoxhunt training, practice makes perfect; or, more accurately, practice makes resilient.

Over time, individual skill levels continuously rise and organizational risk levels decline. When people stay engaged with training, they get better and better at recognizing and reporting phishing simulations--even though the phishing simulations themselves are designed through our adaptive learning AI to get more challenging.  

Neglected phishing simulations represent the biggest unknown in human risk. When people aren't participating in a training program, they aren't learning. It can't be assumed that a missed phishing simulation is a good thing, as most traditional failure-focused training programs do. In fact, our data indicates that high miss rates predict higher risk of a breach.

This is huge. But let's take a step back and tie this information it into context.

Focusing on failure is a failed approach

When companies begin with Hoxhunt, their baseline metrics are typically a failure rate of 25%, a success rate of 4%, and the rest are missed. That's a big unknown.

Traditional training offers one metric: failure rate. When someone clicks on a phishing simulation, they are scolded about their failure. They are told to, at best, go through dry contextual training on their screen (which few do when they are only punished for failure, and never rewarded for success); or go through additional training or, at worst, get publicly humiliated, go to phishing prison, or even get fired (we're not making any of this up).

But this failure-focused approach is doomed to fail.

Meaningful metrics: the 5 vital statistics of your organization's human risk

Fixating on failures leads to failure. Focus on 'misses'. Realistic practice never stops paying

A phishing training program has 5 vital statistics. They must track

  • Failure rate: the portion of phishing simulations people click erroneously
  • Miss rate: The phishing simulations that they neglect for whatever reason.
  • Success rate: The phishing simulations that are correctly reported
  • Real threat reporting: The number of real phishing attacks- per-user that get reported
  • Engagement rate: the proportion of the organization who are enrolled and participating

Meaningful metrics: How big is your unknown?
In our analysis of millions of user responses to phishing simulations, fail rate had the lowest correlation to real threat reporting than any other metric. Whether the fail rate was low or high, it did a poor job predicting whether people would actually spot and reoprt the real thing.
Meaningful metrics: How big is your unkown?

Failure alone is just the tip of the iceberg. To be focused on failure is to be blind to the true risk surrounding your employees and your organization. Failure rate does not shine a light into your unknown risk--your people's skill, or lack thereof, at recognizing and responding securely to a phishing attack.

Solving the mystery of misses with success

With Hoxhunt, rewarding success is foundational to training. Threat reporting is built into the program with a simple button, which triggers a positive learning experience in the form of feel-food contextual training followed by gold stars and a rise in shields rankings along a gamified journey of awareness. The idea is to make threat hunting and reporting fun and rewarding, so it becomes a reflex.

Positive reinforcement works. Just ask DocuSign's Director of Trust & Security Training & Awareness, Lisa Kubicki. She did a whole webinar on how she uses Hoxhunt to advance her own training methodology that's based on multiple principles of behavioral psychology. Check it out here.

But how do we see how training is working in the real world? By real threat reporting rates.

Our data shows a powerful correlation between the number of real threat reports by users and their miss rate and success rate.

The best outcome in a real phishing attack is a threat report. High success rates and low miss rates are the best predictors that an employee will hit the threat report button when a real phish lands in their inbox.

Organizations will typically start with real threat reporting rates at around 0 per user, per month. Over time, as miss rates decline and success rates rise, real threat reporting rates will go to around 1, and as high as 2 per user per month.

When a real threat is reported it is removed from the system, thereby protecting fellow employees from the danger.

A miss today is a phish tomorrow

No one else has ever depicted the vital signs of behavioral cybersecurity in such a simple, crystal-clear snapshot. Bottom line is, this is good news. And it's supported by millions of good, hard datapoints. When you focus on your misses instead of failure, good things happen.

When you focus on your misses instead of failure, good things happen

Behavioral Cybersecurity Statistics Report

For more information, make sure to read the Hoxhunt Behavioral Cybersecurity Statistics report. It contains unique insights into how 1.6 people from all sorts of backgrounds interacted with phishing attacks and 24.7 million phishing simulations.

Want to learn more?
Be sure to check out these articles recommended by the author:
Get more cybersecurity insights like this