The Teachable Moment?

Posted by Mike Broering—Vice President of Sales 


A common feature of most, if not all, anti-phishing behavior management programs is the use of “teachable moments.” If you are unfamiliar, a teachable moment is delivered during a phishing awareness campaign and politely informs the user they did something wrong. The “teachable moment” is provided to the user on the landing page and points out all of the telltale phishing signs they missed and how to (hopefully) avoid similar errors in judgment in the future.

As a leading provider of anti-phishing testing and learning solutions, PhishLine incorporates many teachable moments into our phishing and social-engineering simulations. But for just a moment, let’s consider the other side of the teachable moment. Is the user in a Teachable Mood?

So what happens when some users hit a landing page that says, “You’ve been Phished!”? For many, they quickly look over their shoulder and promptly close the browser page, feeling silly or upset for clicking. Our data shows that when a user is offered a short training video to help them recognize phishing attempts in the future, sometimes less than 10% of users take the training.

If the magical in-the-moment training is not the silver bullet some would lead you to believe it is, how do you achieve meaningful results? Here are a few suggestions:

  • Before you throw the baby out with the bathwater, conduct a test to see if in-the-moment teaching is effective for your usersRun an A/B test in which one set of users is offered in-the-moment training and another set of users is unaware they were part of a phishing test, delivering training separately post test to those who clicked. Then retest. Which group performed better?   Was there a statistical difference?
  • Conduct phishing and other social-engineering tests to measure user behavior and better understand the exact problem you are trying to solve. Do some users avoid the link, but fall down when there is an attachment? Do others respond to the email, thereby opening communication between the user and the “attacker”? Understand the behaviors you are trying to change at a deeper level and then determine the best method for remediation and learning. Apply the training that addresses the specific behavior the user or group of users need work on.
  • Test. Test. And then test some more. Recognize that the training you are delivering today may show results now, but that may not be the case six or nine months later. Users begin to tune out repetitive training and thus effectiveness can wane over time. In some ways, changing user behavior is a lot like marketing. We are trying to get a user to do something (or stop doing something). Direct marketing firms use A/B testing in direct mail pieces. The control is the piece of mail sent to millions, soliciting users to donate or buy something. A test piece is then sent to a smaller (but statistically significant) number of households to determine if the test piece out performs the control. If it does, it becomes the control and they begin testing against it. Security awareness training should be thought of in the same manner. Provide training and follow-up with testing to determine if a better message or communication path produces better results.

Testing and training is a continual process. If you follow a Plan, Train, Test, Measure, and Take Action process, you will continue to make meaningful progress.

Related posts