Phishing Education - Maybe 'Best Practice' is not Best After all

As security practitioners and leaders, we must contribute to the professionalization of our field by searching out data and evidence-based solutions for our organizations.  For too long, we have been advocating solutions based on best practice, or “That is just the way it is done.”   As time goes on, when studies are done, we often find that the results do not back those conclusions, and we must change.

Last week at Blackhat, a group of researchers released the results of their study into the efficacy of phishing training.  You can read that study here.  By applying medical research standards to the problem, they ran an 8-month experiment, randomly separated a significant sample size of over 19,000 employees into a control group receiving no training and four other groups receiving the most common methodologies of security training today.

The results of the research are interesting and should give us all lots to think about.  The key points I pulled out are here:

1. The person who creates the lure message controls the failure rate.  The better the lure, the more people fail.  This leaves the results open to manipulation from the person controlling the lure.
2. Over the 8 months, most of the users failed at least once, leading to the conclusion that over a sufficiently long period, most people will click on something.
3. The frequency of cybersecurity training had no observable benefit.  The percentage of users who failed the test was not markedly different for those who had the training that day or those who had received the training more than a year ago.  The overall average improvement was 1.7% for those who did the training vs the control.
4. In some cases, the training was counterproductive, with employees being slightly more likely to click on the lures than the control group.

(Ho et al., 2025)

Now, this does not mean we should not train our users, but the data indicates we cannot train them out of this problem.  So, what should we do?  I propose that our industry focus on a couple of key areas:

• Security engineering – A user being human and clicking on a link should not, in and of itself, cause a material breach.
• Security training must focus on teaching users to report.  1% of phishing recipients reporting phishing to your team will be more effective than the sub-10% click rate you brag about to your management team.  Reward your reporters, get rid of the wall of shame, and replace it with a wall of fame. Send them gift cards (real ones, not phishing tests) and celebrate them for the cyber heroes they are.

By getting away from the game of “gotcha” that we have built into our phishing training programs, we will also realize another happy side effect.  Trust.  In any organization, cybersecurity advances at the speed of trust.  We don’t build trust with users by fooling, tricking, or making them feel dumb.  We build trust by supporting, showing empathy, and helping them reach their business goals.  For too long, phishing training has been an obstacle to building trust.  It is long past time to put gotcha-based phishing training and simulation to pasture – now we have the data that proves it is as ineffective as it is damaging.