Phishing simulations

Here are some references I use to support my negative view of Phishing Simulation [1]:

Perspectives:

• Sean Cassidy: Phishing simulations considered harmful
• Jamie Finnigan: Simulated phishing is not so great
• Yahoo Paranoids: Stop Giving Impossible Advice: Telling People to Watch Out for SUSPICIOUS EMAILS is Nonsense.
• Jacob Kaplan-Moss: Don’t include social engineering in penetration tests
• Matt Linton: On Fire Drills and Phishing Tests

Research:

• Phishing in Organizations: Findings from a Large-Scale and Long-Term Study
• “What Keeps People Secure is That They Met The Security Team”: Deconstructing Drivers And Goals of Organizational Security Awareness
• “To Do This Properly, You Need More Resources”: The Hidden Costs of Introducing Simulated Phishing Campaigns
• “Employees Who Don’t Accept the Time Security Takes Are Not Aware Enough”: The CISO View of Human-Centred Security
• NDSS 2024 - Symposium on Usable Security and Privacy
  • Keynote - Dr M Angela Sasse
  • The Impact of Workload on Phishing Susceptibility: An Experiment
• What Motivates and Discourages Employees in Phishing Interventions: An Exploration of Expectancy-Value Theory: "Our study reveals a spectrum of factors that influence employees’ intentions to report phishing emails. ... Among the factors discouraging employees, the absence of feedback and perceived low utility value are particularly detrimental."
• Understanding the Efficacy of Phishing Training in Practice: "Taken together, our results suggest that anti-phishing training programs, in their current and commonly deployed forms, are unlikely to offer significant practical value in reducing phishing risks."

Commentary:

• Good phishing simulations are dependant on a good lure. Meaningful lures involve major threats or inticements, both of which can hurt the end user when followed by the "gotcha" page.
• Phishing simulations may be needed to check a box for legal, contractual, or compliance requirements. It's up to you to fight the first two getting put in place.
• Measuring clicks is bad simulation - we mostly care about credential submission or execution of a binary. Simulation vendors often over-emphasize clicks as "bad."
• Consider gamifying phish detection, as one alternative
• Phil Venables: Security Training & Awareness - 10 Essential Techniques
• Defense in Depth podcast: Are Phishing Tests Helping or Hurting Our Security Program?

[1] Instead, roll out webauthn/Yubikeys