Not All Clicks Are Created Equal: Addressing 2 Major Challenges to Your Social Engineering Program

Posted by Mark Chapman—President & CEO of PhishLine • 2/25/2016


Beyond the basics, such as themes, frequency of tests, and other common issues, the two biggest corporate cultural challenges faced when introducing Social Engineering programs into an organization are misguided metrics and too much focus on a small number of solution choke points.  Understanding and learning how to work around these challenges will help you create a program that truly benefits your company.

Many information security awareness programs become obsessed with one – and only one – metric, the “click through rate.” This easy-to-understand metric represents the percent of people who clicked a link on a particular campaign email. To be fair, much of the marketing in this industry historically focused on reducing click through rates.

This myopic view ignores the fact that not all clicks are created equal from a risk perspective. If a user clicks on a link, then submits a login and password, and then uploads a budget spreadsheet, is that the same risk than if they just clicked on a link and closed a landing page?   Are there other risky actions a user can take other than clicking?   Many attackers are using attachments and other techniques to perform social engineering.   You can learn a lot by performing a mock phishing campaign that has no links whatsoever.   The call to action can be to solicit an email reply or to direct the user to a different vector, such as calling a phone number.

At the end of the day, whether it is “click through rate” or some variation of “engagement rate,” there are other fundamental problems with focusing on a raw metric. Unless you are just repeating slight variations of the same campaigns, the theme, design, timing, and other factors can affect click through rate more than any specific remediation.   This can lead to frustration and disillusionment with respect for the overall value of the program. “Our goal was to get click rates down to 10%, but they keep bouncing around every month.”

The second biggest challenge is too much focus on a small number of solution choke points. Early on, programs tend to exclusively focus on the “teachable moment.”  While the teachable moment holds high value, it is not the only purpose of a good social engineering awareness program.  The most effective program will help you discover and measure risk across your user base, then target the right training and threat mitigation strategies to the individual groups who need them. 

The bottom line is to ask yourself:   

  1. How does this program affect our real-world security posture?
  2. Are my users just being conditioned to recognize mock phishing exercises?
  3. Is the program accidentally designed to artificially limit the types of mock phishing exercises to ensure improvements to the metrics?

One of our customers made the analogy that measuring the success of a security awareness program by only looking at raw click through rates is just like measuring the success of a patch management program by looking at the number of new vulnerabilities found each month.

Metrics must be contextual, relevant, and actionable. For a vulnerability management solution, the raw number of patches is much less interesting than the average-time-to-patch.

For a phishing program, putting raw click through rates in the context of hypothesis based testing and risk-based observations will allow much better, more actionable metrics.   “We learned this, we validated that, we discovered this, and we adjusted that.”

And remember, at the basic level, everyone initially struggles with selecting appropriate themes for the mock social engineering exercises. The frequency of tests, the sophistication of tests, the pre-announcement of tests, the number of people to test, what level of results to share, and other common issues need to be addressed.   If you are struggling with these issues, the PhishLine Information Security Awareness Operational Planning™ (ISA-OP) methodology and our experienced consultants can quickly work through these typical issues while defining campaign planning that identifies clear objectives, stakeholders, constraints, and recommended campaign content.

Related posts