Executive Summary
The biggest human cyber-risk is neglecting your humans. Through that fog of uncertainty, social engineers make billions by being better than the good guys at knowing what makes people tick and, accordingly, how to make them click.
So, who's clicking on what, and why? And how accurately and quickly are employees reporting threats?
Categorized by geography, job role, phishing theme, and industry verticals, this report provides fresh insights into employee cyber behavior. It reveals never-before-reported metrics and connections between the changing cybersecurity landscape, real employee threat detection, and their cybersecurity training. It includes, for the first time, the element of speed, or dwell time, alongside real threat detection.
Best of all? This report offers hope for security teams. The global averages of cyber behaviors after 1 year of Hoxhunt's security awareness training for employees prove that good training indisputably improves behaviors and reduces human cybersecurity risk. People really can make a difference. Understanding where your greatest sources of human risk are lets you channel resources to the right people in the right place and at the right time.
Cybersecurity is a team sport. The data from this report shows that we can build a strong security culture by creating a psychologically safe environment where individuals are rewarded for success and coached on mistakes. A collective effort encourages users to feel personally responsible for security and fortifies an organization’s cyber defenses.”
– Petri Kuivala, CISO Advisor to Hoxhunt & Former CISO of Nokia and NXP
Report methodology & key terms
This report is based on data collected from 15 million Hoxhunt phishing simulations, and millions of real reported emails, sent to 1.6 million users in 125 countries.
As a result, it offers statistically significant results that security leaders and CISOs can pull insights from to shape their security training programs and secure the budget they deserve.
To effectively explore this report, you'll need to be familiar with the following terms:
- Success rate: Correctly reporting a phishing simulation
- Miss rate: Neglecting to report or click a phishing simulation
- Real cyber threat detection: Reporting a real phishing email
- Failure rate: Clicking a phishing simulation link
- Dwell time: Time between receiving and reporting a phishing email
- Onboarded: Enrolled in the Hoxhunt program
Understanding the Threat Landscape: Macro Trends
To understand the impact of security awareness & phishing training, we first need to look at the macro trends.
In 2023, the volume of ransomware, BEC, and phishing attacks skyrocketed:
- 1265% increase in phishing attacks in 2023 (Slashnext State of Phish)
- 100% increase in BEC attacks (Verizon DBIR)
- 51% increase in ransomware attacks on large enterprise (CrowdStrike)
- 50% increase in ransomware payouts (Chainalysis)
Black hat generative AI tools made dark web phishing kits and ransomware-as-a-service products cheaper and more sophisticated at scale, which resulted in a deluge of phishing emails full of ransomware, credential harvesters, BEC attacks, and other assorted malware.
"Generative AI has raised the danger of phishing attacks, but it’s also raised the level of phishing training, too.Our research proves that good training protects against evil AI."
– Pyry Åvist, CTO & Co-Founder at Hoxhunt
Phishing costs & time
According to the Ponemon Cost of Phishing study, the average annual cost of phishing more than tripled between 2015 and 2021.
The 2023 IBM / Ponemon Cost of a Data Breach study found a $1.2 million difference between breaches that were identified and contained before or after 200 days of initiation.
It also found that poorly trained vs. well-trained employees were the biggest cost-amplifiers and cost-mitigating factors. Speed and skill in cybersecurity behavior can save companies millions.
"Some individuals will always make mistakes. Our mission is to build resilient communities where vigilant security champions have the skills and tools to report attacks fast and keep their colleagues safe. The larger this population is, the less those companies have to worry.”
– Petri Kuivala, CISO Advisor to Hoxhunt & Former CISO of Nokia and NXP
Security Training Performance by Region
Training performance may vary in different places. Countries contain different business and cultural norms, and therefore certain training approaches or types of phishing attacks may thus perform better or worse. The correlation between success rates and miss rates further validates findings by Hoxhunt that reporting a phishing simulation – what Hoxhunt terms, a Success – is the key metric for tracking and unlocking resilience.
The table below shows the highest and lowest performing countries based on Success, Miss, and Failure rates, sorted by highest Success rate to lowest.
Security training performance by country
Security training performance by continent
"When leading cybersecurity awareness programs for globally distributed teams and companies, understanding the differences in behaviours between units (country, department, subsidiary etc.) of an organization is key. Once I have this data, I tend to start where the engagement is the lowest (ie. where I see the highest miss rate), and connect with local colleagues to understand how cultural factors and local environments can affect their attitudes and behaviours, and how we can together find solution to improve engagement. A one-size-fits-all approach fits none, so we need to meet people where they’re at.”
– Maxime Cartier, Head of Human Risk at Hoxhunt; former Head of Security Culture & Competence for H&M Group
Trends by Region
- Europe is the highest-performing region globally, by a significant margin
- China’s poor performance across the board could reflect both technical as well as cultural cybersecurity training challenges, e.g. The Great Firewall of China
- France has amongst the lowest performances amongst European nations, but Switzerland is the highest. With both French and German-speaking users in Switzerland, language is likely not the issue
- Germany and Austria are both amongst the top performing countries and share a language and perhaps
other cultural attributes
“I lived and worked as a CISO in China and my strong assumption is that the culture of ‘losing face’ drives inaction, as seen with the high miss rate. When the miss rate is high, people will not learn. Hence the fail rate is also higher with the cultures where reporting is not instinctual. This cultural norm is not good or bad, but bears keeping in mind when designing an awareness training program with cultural targeting.”
– Petri Kuivala, CISO Advisor to Hoxhunt & Former CISO of Nokia and NXP
Security Training Performance by Industry
Different industries are attacked at varying levels of intensity. This is likely due to malicious actors seeking high ROI in their attacks. Industries that yield the most profit with the least effort for a cyberattack, perhaps due to lower level of security maturity and poorly trained employees, will look like easy prey.
The table below shows the highest and lowest performing industries based on Success, Miss, and Failure rates, sorted by highest Success rate to lowest.
Trends by Industry
- Manufacturing & construction is the most targeted vertical (over 300,000 attacks per organization in 2023), whereas the tourism industry seems to interest attackers the least (less than 500 attacks per organization in 2023).
- The high threat reporting activity in the financial services (61%) and utilities (50%) sectors are encouraging and, respectively, 100% and 66% higher than the lowest performing industry, Retail (30%). This may be due partly to there being more computer-based work in Finance and Utilities than in Retail, along with a stronger security culture in sectors that have long been prime targets for bad actors.
- Also note the poor success rate in Pharma & Healthcare, an industry particularly challenged by phishing and a very busy workforce.
- Meanwhile, the Retail and Logistics sectors' low failure rates are offset by their low success rates and high miss rates, indicating higher uncertainty in human cyber security risk.
- Pharma and healthcare's second-highest miss rate (54%) is concerning given the FBI's IC3 report naming it as the critical infrastructure sector that's most breached by ransomware, and that it's experienced the costliest average data breaches for 13years in a row (IBM Cost of a Data Breach). At $10.93 million, healthcare is almost double the number two sector,Finance at $5.9 million, and over twice the next sector, Pharma, at $4.82 million.
”Financial Services is the industry with the highest performance by a large margin. This is perhaps unsurprising considering this is an industry that has historically had a strong security culture, with targeted regulations and attacks giving companies a clear incentive to protect money from ending up in the wrong hands. Healthcare and retail come at the bottom, mostly because a significant portion of employees in these sectors are front-line workers who need to spend most of their time with their patients or their customers, not on a computer.”
Maxime Cartier – Head of Human Risk at Hoxhunt; formerHead of Security Culture & Competence for H&M Group
Security Training Performance by Job Function
Different job roles contain different levels of access to sensitive information as well as different types of computer use and communications. Moreover, certain job roles like IT are typically filled by people with higher or lower levels of computer and security knowledge. Threat actors target people based on the types of communications they are accustomed to receiving.
The table below shows the highest and lowest performing job functions based on Success, Miss, and Failure rates, sorted by highest Success rate to lowest.
*Readers may notice an apparent data discrepancy between the job function dataset and the geography and industry datasets, in that the success rates for this dataset is higher. This is due to the fact that users self-report their job role after several months in training. Job function data doesn't include inactive users so, lacking the inactive users, this cohort's performance rates appear better.
Trends by Job Function
- Legal and Communications have the highest and lowest respective success rates. As expected, legal, finance, and IT departments are high reporters while comms, sales, and marketing are lower performers.
- Communications and business development have the highest failure rates, with communications’ failure rate being 40% higher than Finance, the top performing category.
- Direct correlation between job functions’ success and miss rates. Comms, sales, business development and marketing tend to have more spam and email to go through than other departments, which could contribute the poor performance.
- As with the finance industry in general being historically highly targeted by attackers, the finance department is attractive to criminals due to its access to money. Thus, they can receive added security training to protect the bank vault, with stronger security incentives and processes. It’s great news that finance department professionals are amongst the best at reporting and not clicking.
"I always take into account the type of workpeople do when I’m designing a security awareness curriculum. For instance, frontline workers in healthcare and retail need practical training. They are very busy, and constantly switching between bursts of computer work and human contact.They need short, relevant training content that mimics and addresses the attacks and issues they’re facing."
– Maxime Cartier, Head of Human Risk at Hoxhunt; formerHead of Security Culture & Competence for H&M Group
Security Training Performance by Phishing Theme
Social engineers target people so training should, too. The variance in the effectiveness of different types of phishing attacks reminds us that a cookie cutter approach to training is not optimal. Training can be tailored to take into account that click rates vary by industry on different themes.
The table below shows the highest and lowest performing phishing themes based on Success rate, Miss rate, and Failure rate, sorted by Failure rate.
Phishing Type Glossary
- Invoice scam: Fraudulent invoice demanding action be taken on a payment.
- IT Admin (inter org): Phishing email issued from the company’s IT department internally.
- Authority impersonation: Impersonated authorities include organizations like: tax office / healthcare / parking lot company / bank / known brands, etc.. It also includes work-related invoice scams.
- Personal: Only applicable to the receiver. Uses sender-familiar language and mentions items specific to the receiver.
- Sensitive information gathering: Attempts to steal sensitive data via phishing lures like: register here, update info here, etc.
- Temporal attacks: Phishing emails that are only valid at a certain time of the year, e.g.: summer, holiday, Black Friday, Christmas, Easter, etc.
- Dangerous files: Phishing email containing fake or real attachment, or file share.
- Inter org communications: Work-related phishing email spoofing the company but not necessarily the IT department. It can come from e.g. your co-worker, CEO, HR manager.
- Email environment: Email client-specific phishing email spoofing Outlook or Gmail.
- Online services: Phishing email that resembles a real service (logo, brand, colors). Can be a fake brand.
- Packet delivery notifications: Classic phishing emails spoofing services like UPS / USPS / DHL and local carriers.
Trends by Phishing Type
- Most industries click most often on Inter-org communications, like spoofed HR comms. The 7.4% global failure rate tracks with social engineers’ tendency to pose as a colleague to lower the barrier to trust.
- Invoice scams are second most click on at 5.7%, although these are also the highest reported simulations at 67%, over twice the success rate of packet delivery notifications at 32%.
- Scam packet delivery notifications are a common phishing lure. While the failure rate is low here, so is the Success rate. It’s likely people delete these often, but they should be taught to report them.
- Dangerous file downloads are the third-highest clicked (4.6%) and fifth-least-reported phishing simulation. This is a common delivery method of the worst forms of malware, such as ransomware.
QR phishing attacks surge
2023 was the year of QR phishing (or "Quishing"), as attacks surged by 22x. Nearly non-existent in 2022, QR phishing attacks comprised roughly one quarter (22%) of all attacks on our user network by October 2023. While many know the risks of clicking on a suspicious link or file, fewer are aware that QR codes can also deliver malware or credential harvesters.
In a benchmark study of nearly 600,000 employees in 125 countries, Hoxhunt found that just over one-third (36%) of recipients successfully identified and reported the simulated attack, while nearly 2/3 missed it (59%), and 5.5% of employees failed, either scanning the QR code or clicking the link. Interestingly, over 2.5 times as many users clicked a malicious link in the traditional way than scanned the simulated malicious QR code: 3.9% clicked, and 1.6% scanned.
Attackers likely find QR phishing attractive because they can bypass email filters more effectively, although that trend appears to have reversed as email gateways have hardened their defenses against malicious QR codes.
Global Impact of Phishing Training for Employees
The data after implementing Hoxhunt shows indisputable global improvements in every relevant metric, from simulated phishing failure and success rates, to dwell time and real threat detection:
- 9x threat reporting: this rise in threat reporting is not surprising given that most programs neglect threat reporting and most users lack the knowledge, skills, and tools to do so
- Median dwell time 1/3 faster: dwell time is the time elapsed between a phishing email landing in an inbox and a user reporting it. In cybersecurity, time is essential for mitigating risk
- Fastest 5% report threats in 39s: the fastest 5% of reporters keep the whole organization safe because they can accelerate SOC response to a threat in under one minute of its appearance, eliminating the threat before others can click on it
- 2/3 report a real threat within first year: Never before have we seen the element of speed in cyber behavior metrics, distinguished by user percentile of performance and in both the training and real world contexts
Performance improvements over time
The standard, pre-Hoxhunt SAT performance baseline is 7% Success, 20% Failure, and 80% Miss rates. Many enterprise organizations with legacy SAT models often have stagnant Success rates of about 10%, with limited visibility into real threat reporting and dwell time.
These metrics all drastically improve once onboarded with Hoxhunt and steadily improve over time, demonstrating sustainable engagement and resilience.
Note how the Success rate and Miss rate trend lines invert at onboarding and continue to steadily improve over time.
Meanwhile, the Failure rate plummets from 20% at the estimated baseline to 8.7% at onboarding, and continues to steadily reduce over time. It flatlines around 2% after 2 years, showing the importance of frequent, continuous training over time.
The pre-Hoxhunt baseline is an estimate based on multiple data points, including:
- Analysis of hundreds of thousands of users segmented by the frequency of their training program into a SAT-standard quarterly curriculum
- Customer surveys and POC data comparing Hoxhunt to established SAT tools like KnowBe4, Cofense, Proofpoint, etc.
Actual numbers can vary between companies significantly depending on:
- The difficulty level of their phishing simulations
- The frequency of their phishing simulations
- The quality of training and program content
"With phishing simulation engagement rates reaching above 60 percent and failure rates dropping below 2 percent, Hoxhunt has helped us push our resilience into new territory, and surpass anything our legacy SAT tools could deliver.”
– Ryan Boulais, VP & Chief Information Security Officer at AES
Bridging the gap between training and real threat detection
In addition to simulated threat reporting, Hoxhunt reframes security awareness, behavior, and culture programs around a new set of metrics: real threat detection. This is a landmark departure from the old-school Security Awareness Training (SAT) model’s simplistic dependency on simulated phishing link click rates.
With Hoxhunt, the amount of real threats reported also grows steadily over time, showing the correlation of training having real-world impact.
Understanding your people, and your approach to security training's impact on them, starts with measuring the cyber behaviors you want to improve. Phishing doesn't stop at training, so neither can the metrics. The ideal outcome of a phishing attack is that it gets reported so SOC response can remove it from the system. Thus the most effective behavior to monitor is threat reporting.
Hoxhunt’s gamified phishing training platform uses an adaptive learning model to create personalized learning journeys for individuals. People practice recognizing and reporting phishing simulations until resilience becomes a reflex.
Then, Hoxhunt is designed to extend and connect training to measurable real threat detection outcomes.
People report potential threats the same way that they are taught to in training. Hoxhunt uses AI to categorize reported real threats in real-time. In-the-moment feedback and gamified rewards are provided to users when they report a real suspicious email, which reinforces that behavior.
All of this becomes a critical part of security behavior and culture programs, according to Gartner:
“New capabilities are emerging to meet the demand for improved human risk management. These security behavior and culture programs (SBCP) capabilities focus on risk reduction via tangible employee behavior management. Innovative solutions build their services based on behavioral science principles, and use data analytics and automation to reduce risk exposure via measurable culture change.”
– Gartner: Innovation Insight on Security Behaviour and Culture Program Capabilities, Nov. 16, 2022
Hoxhunt provides capabilities that go beyond the traditional SAT model, including some or all of:
- AI-enabled adaptive learning
- Automated training operations + threat data orchestration
- Behavioral science-driven curriculum design
- Game mechanics
- Reward-based, personalized learning journeys
- Real threat detection monitoring, with in-the-moment feedback and micro-training
- Measurements of dwell time
Conclusion
While the stats and trends around the threat landscape and human cyber risk are typically grim, this report offers hope. It indicates that, regardless of employee industry, background, or location, risk can be reduced with ongoing participation in a gamified, adaptive training program.
The importance of connecting and revealing dwell time and threat detection from the training to the real world context can’t be overstated. We equate risk with real threat detection. A threat report reduces risk more than any other action. The faster it is submitted and responded to, the less damage the social engineering attack will cause.
The correlation between success rates and miss rates further validates findings by Hoxhunt that engagement is the key metric for tracking and unlocking resilience. Inactive employees aren’t learning or reinforcing secure behaviors, so misses are more likely to become phishes.
The variations between the different user cohorts’ phishing outcomes underscore the need for tailored training and targeted interventions based on risk profiles. By knowing our people, we can keep ourselves safer from the bad guys.
"I’m so confident in our staff now with Hoxhunt that if people ask me how many cybersecurity officers I’ve got, I say ‘2000.’I know that everybody is going to be reporting threats and doing their job. We’re flipping that human layer from being the biggest weakness to the biggest strength.”
– Mark Sedman, Global Head of Cybersecurity, WaterAid
