Security Awareness and Phishing Simulation That Works

Phishing is an attempt to trick someone, by email or message, into clicking a link, opening a file, or entering a password. Security awareness is the set of activities that teach staff to recognize this kind of threat and to know what to do when they meet it. A phishing simulation is a fake phishing email that an organization sends to its own staff, to practice and to measure how well they cope. This guide shows how to run a security awareness and phishing simulation program that works, by measuring who reports as well as who clicks, and by never punishing people who slip.

Phishing defense has four layers

The NCSC (the UK National Cyber Security Centre) splits phishing defense into four layers in its guidance Phishing attacks: defending your organisation. Training staff is only the second layer. The NCSC says training, especially phishing simulation, is the layer that is often over-emphasised, because no training can teach everyone to spot every phishing email, and some phishing will always get through. So an organization must also plan to limit the damage. If it cannot do everything, the NCSC advises doing at least some measures in every layer.

• Layer 1, make it hard to reach users: use DMARC (a policy saying how emails that fake your domain should be handled), SPF (a list of servers allowed to send email for your domain), and DKIM (a digital signature proving an email really comes from the domain) to stop sender spoofing, filter malicious email and spam, and reduce the organizational information attackers can use to deceive
• Layer 2, help users notice and report: train staff, change work processes so fake requests are easier to spot, and build a culture where reporting is easy
• Layer 3, limit the damage when someone slips: protect devices from malware with patching and limited admin rights, block malicious websites, and use MFA (multi-factor authentication) and least privilege
• Layer 4, respond to incidents quickly: keep logs so incidents are detected early, and have a response plan that has been rehearsed

The NCSC gives a real case study of a financial company with about 4,000 employees. It received 1,800 emails carrying variants of Dridex malware, posing as an invoice that needed urgent attention. The email filter stopped 1,750, leaving 50 that reached user inboxes. Users clicked 14 of those. The malware failed to launch in 13 cases because devices were up to date, and in 1 case it installed, but the malware's call back to its operator was detected and blocked, and the device was seized, investigated, and cleaned within a few hours. Users also reported 25 emails in total (including some reported after the click), and the NCSC says these reports were the first sign that the attack had passed the first layer. Its conclusion is that relying on any single layer would have missed some of the attacks.

The NIST approach: a program that improves in cycles

NIST SP 800-50 Rev. 1, Building a Cybersecurity and Privacy Learning Program, is a final publication (September 2024), after two public drafts (September 2021 and August 2023). As of October 2026 the NIST page still lists it as final. It supersedes SP 800-50 (2003) and SP 800-16 (1998). It is written for US federal agencies, but NIST says it is meant to suit both large and small organizations. In it, awareness is an activity carried out on an ongoing basis throughout the year, not a once-a-year course, and the goal is that staff know their roles and can take appropriate action.

NIST divides the work of the program into four phases that repeat, and says the phases can run in sequence or at the same time. The four phases are:

• Plan and Strategy: tie the program's goals to the organization's goals and risks
• Analysis and Design: work out who needs to learn what, and split the audience into groups, for example general staff, leaders, and system administrators
• Development and Implementation: build the content, awareness activities, and practice exercises, such as phishing simulations, and put them into use
• Assessment and Improvement: measure results, check whether behavior really changed, and feed the findings back into the plan

NIST treats phishing simulation as experiential learning (learning by practicing on realistic scenarios) that can target everyone or specific groups such as leaders and administrators. It advises starting from the current click rates of each group to see which groups need more learning support. It states plainly that these exercises should not be punitive and that no employee should be called out for their response. The same idea of simulating attacks to find weaknesses applies to AI systems, see AI Red Teaming: How to Test AI System Security.

How to run a phishing simulation, step by step

The steps below are drawn from NCSC and NIST guidance, and the ordering is our own. The steps are:

• Talk to HR (human resources) and legal first: the NCSC says always to check with HR before running phishing simulations, and NIST says to involve the legal team in designing and reviewing the plan, for example avoiding real brands or named organizations in the bait, because people might then phone or email those organizations
• Tell staff in advance: say that the organization will run exercises from time to time on a random basis, and that results will be used to guide learning, not to punish (as NIST advises)
• Measure a baseline and prepare reporting: a baseline is the starting figure, taken before you change anything, used for comparison later. Record both click rate and reporting rate for each group, and make reporting easy before sending any practice email, for example a report button in the email client, which NIST gives as an example
• Design the practice email and rate its difficulty: make the email fit the real work of the target group, and use the Phish Scale (a NIST method that rates how hard a phishing email is to detect) to record the difficulty of each one
• Send, and answer every report quickly: the NCSC advises giving quick feedback on what action was taken on a report, and showing that staff reports make a difference
• Analyze by group and adjust learning: see which groups need more help. If click rates are higher than usual, NIST suggests repeating the message through other channels

Measure without fooling yourself

A phishing simulation produces clear numbers, which is why it is popular, but the NCSC warns that metrics express an organization's values. If the organization seems to value the absence of reports of problems, it encourages people to stay quiet. So the NCSC advises measuring how many people reported, alongside how many clicked. NIST likewise gives the share of people who report a test phishing email as an example of a metric for changed behavior. The metrics to look at are:

• Click rate: the number of people who clicked, divided by the number who received the practice email. It should fall, but it cannot be compared across rounds if the emails differ in difficulty
• Reporting rate: the number of people who reported, divided by the number who received the practice email. It should rise, because it shows staff know how to report and dare to report
• Reports made after a click: the NCSC wants people who clicked and then grew suspicious to feel able to come forward, and such reports help the security team learn of an incident early, as in the case study above
• Difficulty of the practice email: the Phish Scale rates difficulty by looking at both the features of the email and the recipient's context, and NIST says it helps explain the click rates of each round

Here is a made-up example, not real data. In the first round, 100 people receive a practice email, 10 click, and 5 report. In the second round the clicks stay the same at 10, but reports rise to 40. Looking at click rate alone, nothing improved. In reality the security team would learn of the threat much sooner, because 40 people raised the alarm instead of 5.

Ethics and law: do not punish people who click

The NCSC says blaming users for clicking links does not work, because people click for many reasons, both personality traits and situations such as being busy or stressed. Threatening punishment does not change these reasons, and people who fear for their jobs will not report mistakes. Phishing simulations can also erode trust between staff and the security team. The NCSC further warns that punishing people for clicking an email the organization itself sent starts to resemble entrapment, which carries legal risk. So do the following:

• No punishment and no naming: as NIST states, these exercises should not be punitive, and no employee should be called out for their response
• Make reporting safe: the NCSC says training should reassure people that reporting will not get them into trouble. This message needs backing from HR, support, and senior management, and people who have already clicked must feel able to come forward
• Use individual data only as needed: ask your legal team or data protection officer how far individual results may be stored and used, and rely mainly on group-level results. This point is our own suggestion
• Consider alternatives: the NCSC notes that some companies ask staff to craft their own phishing emails, or hold a friendly competition, which shows them the techniques attackers use and avoids the feeling that the security team is out to catch them
• Be careful with the bait: avoid topics that play directly on employees' fears or personal interests, such as bonuses or layoffs. This point is our own suggestion and does not come from the sources

What to teach, and how staff report

The NCSC says asking users to examine every email in depth is unrealistic and counter-productive. A better way is to change work processes, for example verifying important email requests a second time through another channel such as a phone call or logging in to the account yourself, and making sure staff know the normal way key tasks are done, such as how payments are made, so they can spot unusual requests. What to teach and prepare is:

• Signs to know: CISA (the US cybersecurity agency) lists strange or unexpected requests, urgent language, and suspicious links. A message can come from a known contact whose account has been taken over, and attackers may use company-specific details to look credible
• Verify outside the message: if a message looks off, do not reply and do not use any number or link in it. Contact the sender through a channel you already have instead (as CISA advises)
• Know in advance where to report: the NCSC notes that if a device is compromised, users may not be able to use normal communications, so they should know in advance whom to report to and how. CISA's Phishing Guidance adds that reporting phishing is one of the most efficient protections, because it helps email providers identify new attacks
• Repeat regularly: CISA's Teach Employees to Avoid Phishing page says once-a-year training is not enough, and suggests naming one person to track new threats and brief staff between training rounds
• Open informal ways to ask: the NCSC suggests channels such as team chat or internal message boards, so staff feel able to "ask out loud" when unsure

For the first layer and for BEC (business email compromise, where an attacker poses as an executive or supplier by email to get a payment made), continue with the article on Email Security: Layered Defense Against Phishing and BEC. Fake voice and video are covered in Deepfake Social Engineering: How Organizations Respond, and the third layer, authentication, in Which MFA Is Safer? SMS, Authenticator Apps, Push, and Passkeys.

Common mistakes

• Punishing or shaming people who click: people become afraid and stop reporting, so the security team learns later
• Measuring only the click rate: ignoring the reporting rate, which is an early warning signal, and encouraging people to stay quiet
• Treating staff training as the whole defense: skipping the email layer, the device and account layer, and the incident response layer
• Comparing click rates across rounds without looking at email difficulty: an easy email and a hard one give very different numbers even when staff are no better or worse
• Training once a year: CISA's advice page says it is not enough, and advises regular reinforcement
• Sending practice emails without talking to HR and legal: both the NCSC and NIST warn about legal risk and the effect on trust
• Nobody answers reports: staff report, hear nothing back, and stop reporting

Start today

• Check your reporting channel today: is there an easy report button or address, who receives reports, and is there a time limit for replying
• Write a one-page rule with HR and legal: no punishment, no naming, results used for learning, and reporting is safe
• Start with one department: send a practice email that is not too hard, measure both click rate and reporting rate, and record them as the baseline
• Set the goal as reporting: do not aim for zero clicks, because the NCSC says some phishing will always get through
• Add a way to verify requests about money and access: require a second confirmation through another channel you already have, such as a call back to a known number
• Review the other NCSC layers: check DMARC, SPF, DKIM, MFA, and patching

To see where the rest of your defenses stand, try the self-assessment for a prioritized view of next steps.

References and further reading

• NCSC: Phishing attacks: defending your organisation: the four layers, its view of training and simulation, reporting, and the case study (published 2018, reviewed 2024)
• NIST SP 800-50 Rev. 1: Building a Cybersecurity and Privacy Learning Program: final, September 2024; see Sections 1.3 (the program life cycle) and 2.7.2 (experiential learning)
• NIST: the Phish Scale and phishing research: the method for rating the difficulty of phishing emails and related papers
• NIST TN 2276: NIST Phish Scale User Guide: the user guide for applying the Phish Scale (2023)
• CISA: Teach Employees to Avoid Phishing: what to teach, verifying outside the message, and regular reinforcement
• CISA: Phishing Guidance: Stopping the Attack Cycle at Phase One: joint guide from CISA, NSA, FBI, and MS-ISAC, published October 18, 2023, with sections for network defenders and for small and medium-sized businesses