case study

How AES, a Fortune 500 global energy company, fueled security vigilance and measurably lowered the human factor in cyber-risks

Client logo
About icon

Industry: Energy, critical infrastructure

Number of employees and contractors: 9,000 +

Challenge icon
Every legacy security awareness training tool fell short of reducing risk and addressing the cybersecurity challenges and needs of AES: a dispersed global workforce; effective scaling of training; supporting multiple languages; perception of security training; low enthusiasm for cybersecurity; poor integrations; and automated analysis of reported threats.
solution icon
Clearly, it was time for a change. Upending the traditional awareness model, AES transitioned to the Hoxhunt human risk management platform and measurably reduced risk and raised resilience by orders of magnitude above the biggest legacy SAT tools on the market.
Key takeaways:
Featured image

Hoxhunt performance vs. AES’s previous security awareness software tools:

  • Reporting (“engagement”) rate 526% up, from the 3-tool aggregate of 11.5% to 60.5%, which reflects the proportion of AES people whose work is computer-based
  • Failure rate 79 % down, from the 3-tool aggregate of 7.6% to 1.6 %  
  • Miss rate 58% down, from the 3-tool aggregate of 80.9% to 34 %
  • Resilience Ratio 2533% up, from RR score of 1.5 to a sterling 38.

The resilience ratio score of 38 is astonishing given the industry, size, and scope of AES. Similar companies will strive for scores of 10-15, and typically topout at 20. The resilience ratio is calculated by dividing the engagement rate by the failure rate, yielding a more accurate metric for risk than either engagement or failure alone.

Ryan Boulais, CISO, and David Badanes, Director of Cybersecurity Strategic Initiatives, talked with us at the CSO50 2023 awards in Arizona about how Hoxhunt fueled their cybersecurity behavior change journey. Learn how they went from stagnant awareness results with old-school SAT tools to CSO50 Award recognition with Hoxhunt's leading edge Human Risk Management platform.

“We focus on engagement. We aren’t beholden to click rate. Previously, we’d had a click rate of 7% with our awareness training solution, but we had a low reporting rate of only 10%. No matter what awareness tool we tried, engagement remained stagnant. We needed a new model to gain better visibility into our human risk and manage that risk, and Hoxhunt enabled that. Now we have a reporting rate of 70% and a click rate of like 2%. We’ve measurably reduced risk and improved security culture in a way that aligns with our cultural values, and people seem to really like it.” — David Badanes, Director of Cybersecurity Strategic Initiatives, AES

Background: “Clearly, it was time for a change.”

Innovation and sustainability are in the corporate DNA of AES. But the security team realized after five years of lackluster results with the biggest SAT tools on the market, that they needed a different approach to reducing risk.

The biggest SAT tools on the market all performed poorly in terms of engagement and risk reduction. This led AES to question the very model of SAT itself, and look beyond to security behavior change and human risk management.

Security training engagement remained stagnant no matter the tool. And If people weren’t participating, they weren’t learning.

82 percent of breaches contain the human element, mostly from phishing attacks. AES understood the importance of protecting their people and company from social engineering and breaches.

“Over five years, we tried multiple solutions and different models with mixed results, navigating from specialized technologies to integrated solutions. We saw modest reductions in click rate, but the engagement was stagnant. Our challenge was not with the solution, but with the security awareness training model itself. Clearly, it was time for a change.” – David Badanes, Director of Cybersecurity Strategic Initiatives, AES

AES found that their current phishing simulation tools and their phishing response model were ineffective, given the current threat landscape, for the following reasons:

  • The one-way awareness training model meant minimal engagement
  • Manual translation for phishing messages was resource-intensive
  • Reliability of phishing metrics were questionable due to the volume of false positives
  • Year-end heightened season for phishing attacks (benefits enrollment, holidays, etc.)

Despite sending more phishing simulations, the threat-reporting / engagement rate declined from 14.1% to 10.6% over 3 years.

hese results are a case study in the "failure" of failure rate, when taken without the context of engagement or the quality of the training itself, to accurately depict an organization's true human risk posture.

AES needed a solution that would help them drive interest in cybersecurity to new levels and make good security behavior a habit. And they hoped to go a step further, integrating human intelligence into threat detection and response. They looked beyond awareness and discovered the Hoxhunt Human Risk Management Platform.

Hoxhunt vs. legacy security awareness training tools

Over one summer, the AES team assessed Hoxhunt with a broad group of users in a defined pilot and created a robust change management process, starting with obtaining leadership and stakeholder buy-in.

Hoxhunt delivered immediate results upon its adoption. These measurable outcomes helped further cement leadership buy-in and further embedded a security culture throughout the organization as a shared responsibility and core business value.

🚀 Engagement skyrocketed

🚀 Risk posture measurably improved

🚀 SOC response to email threats accelerated, without added resources

📈 User skills increased

📈 Failure rate plummeted

📈 Culture lifted

“The end goal is behavior change, and while we originally started with awareness as the focus, I can say we’ve seen a shift in user behavior for the good because people are engaged with Hoxhunt. We like the gamification, we like seeing our place on the leaderboards. I know if I go a week without seeing a Hoxhunt phishing simulation, I go through my inbox and look for them.” — Rodrigo Garcia, Manager, Cyber Threat Management, AES



The human risk management platform delivered gamified, individualized behavior change training at scale and combined several functionalities onto one platform. Compliance, awareness, and behavior change training fed directly into and augmented threat detection and response. It was effectively a self-reinforcing closed loop of protect-detect-respond capabilities.

More training meant fewer clicks and more threat reports. The higher volume of threat reports is automatically orchestrated by the platform to reduce SOC hours and let security leadership focus on the incidents that matter.

“We are able to see the threats a lot faster than we did before and react to them more effectively”  - Rodrigo Garcia

Results with Hoxhunt have been clearly superior to the previous SAT tools. Measurable reduction of risk was achieved after just a few months with Hoxhunt.

The high engagement levels give the AES security team unprecedented visibility into human risk across the organization. This visibility enables a risk-based approach for targeted behavior improvement via:

  • Enrollment in training
  • One-on-one engagement
  • Increased volume of phishing simulations
“We are excited about Hoxhunt overall. The phishing simulations have been very good for the organization. We’ve seen a lot less clicks overall and people are a lot more paranoid about phishing attacks.” - Rodrigo Garcia

Why these results matter

Good security posture is a competitive advantage today. The evolving threat landscape significantly impacts business operations, particularly given the rise in ransomware and software supply chain attacks. As a result, the AES team noted that phishing and training metrics are increasingly requested by external parties, including:

  • Cyber insurance
  • Potential customers
  • Industry analysis (Investor Benchmarks, Compliance Surveys)

AES has outperformed the Hoxhunt global average failure rate despite being a large player in energy and critical infrastructure, amongst the most challenging cybersecurity environments.

Change management: New cyber awareness model

Communication is critical for a behavior change program to thrive. The Hoxhunt platform accelerated the deployment of AES’s new cyber awareness model, to include:

  • Review of a monthly phish in each safety meeting
  • Communication of the leaderboard dashboard and recognition of high performers (star collectors) via:
  • Monthly safety meetings
  • ELT Recognition
  • Safety Day / Values Day
  • Highlight in the Newsletter as “Always on for Digital Safety”
  • Yammer and other internal social media

“The sense of community, the sense of awareness, and the instant feedback on reported real-world phishing attempts are all things people like about Hoxhunt. The gamification, the collecting of stars, and the camaraderie that comes with seeing yourself on a leaderboard have had a positive effect on our culture, which has helped us measurably transform our human risk posture. With phishing simulation engagement rates reaching above 60 percent and failure rates dropping below 2 percent, Hoxhunt has helped us push our resilience into new territory, with our resilience ratio jumping by over 2000 percent in just a few months. Hoxhunt has helped us surpass anything our legacy SAT tools could deliver.” — Ryan Boulais, VP & Chief Information Security Officer

Read and watch the AES CSO50 playbook: 10 steps to award-winning cybersecurity training

The award-winning AES team at CSO50 2023 in Arizona, from left: Rodrigo Garcia, Angie Wyatt, Ryan Boulais, and David Badanes