The recently-released 2023 SANS Institute Security Awareness Report sheds light on a fascinating revelation: while many practitioners cite “lack of budget” and “lack of staffing” as major hurdles, they rarely mention “lack of relevant metrics.”
In my experience, these two aspects are inherently connected. Teams who are focused on check-the-box compliance generally report being saddled with low budgets, and often lack a single FTE (Full-Time Employee) dedicated to security awareness. But even with one person, or less, assigned the challenging task of tackling the human element, it's possible to adopt a security behavior and culture change approach that extends beyond compliance and measurably reduces risk, ultimately helping security teams to obtain additional resources.
Countless courageous colleagues started out as their company’s sole resource dedicated to security awareness. Armed with creativity and enthusiasm as their chief resources, they managed to deliver amazing results! I’ve started this way multiple times myself. But creating and delivering all the training sessions while running phishing simulations and developing a network of security ambassadors, and generally managing stakeholder engagement can quickly become exhausting.
The problem is that the results of typical security awareness and training activities are hard to measure, outside of the number of event participants or viewers on a security video. These engagement metrics are helpful for improving our programs. Security culture metrics are also key indicators to follow-up. For example, the number of people contacting the security team when employees historically tried to bypass it, or the percentage of people who say they would report a security incident.
But these “soft metrics” have little impact on staffing and budget decisions when communicated to executive leadership.
[.c-quote-box][.c-quote-wrapper][.c-quote-icon][.c-quote-icon][.c-quote-right-col][.c-quote-text-wrapper][.c-quote-text]You want to secure more budget? Prove to leadership that your initiatives are effectively reducing real risk.[.c-quote-text][.c-quote-text-wrapper][.c-quote-name-wrapper][.c-quote-name][.c-quote-name][.c-quote-name-wrapper][.c-quote-right-col][.c-quote-wrapper][.c-quote-box]
Measure what matters
It all starts with understanding the human risks facing the business. What are you trying to protect your organization from?
Human risks include social engineering and information storing and sharing, amongst other things, which altogether comprise by far the company’s greatest source of cyber-risk. Identify the attitudes and behaviors that underpin these risk areas, and clearly prioritize which area you want to address first, based on the most common incidents in your organization or external reports. Communicate these risks clearly to the leadership by telling them compelling stories and sharing their associated potential costs—eg. the frequency and cost of a breach via this vector.
Next, develop a strategic set of initiatives to influence these behaviors positively. Measure your baseline levels and after deploying an initiative, measure again to see its effect. Then, communicate your program’s impact in terms of behavior change and risk reduction and, if possible, business value.
For example, for most organizations the #1 human risk might still be phishing. This was the case in an organization I previously worked for. There, by proactively being part of the incident response process, I’ve progressively built my little treasure chest of stories about phishing incidents that had impacted us, and how much they cost. I’ve also spent time listening to employees, to understand why these incidents happened or why they were not reported. In these interviews, I discovered that one of the main reasons behind the low amount of phishing emails detected was that the reporting process was too cumbersome.
The results were astounding
Based on these insights, I launched an experiment: I deployed an easy reporting button in the mailbox for everyone, and a new gamified phishing training for a subset of the employees. The behavior I wanted to achieve was having more people report phishing emails, so that our security team could respond and mitigate the threat as soon as possible. The employees who received training were the action group, the others were the “control group”. After a few months of training, I sent a rather simple phishing simulation email to everyone.
[.c-quote-box][.c-quote-wrapper][.c-quote-icon][.c-quote-icon][.c-quote-right-col][.c-quote-text-wrapper][.c-quote-text]The results were astounding: the people in the training group were 7 times more likely to report the phishing email than employees in the control group! We also observed a significant increase in how many real threats the SOC received, and report relevance kept improving over time.[.c-quote-text][.c-quote-text-wrapper][.c-quote-name-wrapper][.c-quote-name][.c-quote-name][.c-quote-name-wrapper][.c-quote-right-col][.c-quote-wrapper][.c-quote-box]
The final step in this process was to communicate to my leadership the true risk phishing posed to us—with the stories I’d collected previously—and how we were managing it, sharing the results of the experiment. Talking to executive leadership in terms of risks and impactful behavior metrics worked. This was a gradual process, starting with my manager, but it made its way all the way to the C-suite. I strongly believe that having shared tangible metrics about risk reduction is the main reason why I could sensibly increase my budget the year after this experiment—along with a good working relationship with my manager, and explaining what the extra resources would be used for.
Of course, security culture is not achieved with a one-off experiment. And it’s not easy to find the right metric and the right intervention for your specific needs. The SANS report doesn't explicitly list "failure to achieve risk reduction" as a challenge but as John McAlaney, Professor of Psychology at Bournemouth University, told me recently:
[.c-quote-box][.c-quote-wrapper][.c-quote-icon][.c-quote-icon][.c-quote-right-col][.c-quote-text-wrapper][.c-quote-text]Changing people's behaviors is not that difficult – the challenging part is changing them in the direction that you want them to go in![.c-quote-text][.c-quote-text-wrapper][.c-quote-name-wrapper][.c-quote-name][.c-quote-name][.c-quote-name-wrapper][.c-quote-right-col][.c-quote-wrapper][.c-quote-box]
To succeed, one has to be open to experiments and failure. Once you find something that works in your organization, expand it and keep your finger on the pulse! Continuously measure the evolution of attitudes and behaviors over time, and share the most impactful risk reduction metrics with your leaders. It is absolutely crucial in earning the recognition and resources your team deserves. Your hard work will pay off, as the attention of your leaders and a long-term impact on your organization's security will make evident.
About the author
Maxime is a Human Risk Management leader, who has built Security Awareness, Behavior and Culture programs for multiple companies, such as H&M Group. As a consultant at Sopra Steria, he also contributed to security culture programs for global aerospace and manufacturing companies.
A pioneer in this field, he has changed the way people see, talk about and practice security in organisations. Combining deep experience and skills in IT, behavioral science and communications, his goal is to help organisations make the switch from raising awareness to changing behaviors.