Tom Tovar, CEO and co-founder of Appdome
In an era where artificial intelligence (AI) continues to advance at an incredible pace, traditional security awareness training faces more challenges than ever before. The rise of advanced AI-powered threats, including smishing, phishing, deepfakes, and AI chatbot-based attacks, can make traditional human-centric defensive approaches increasingly ineffective.
Today, humans have a slight advantage
Today, security awareness training teaches individuals how to spot the signs and tactics used in social engineering attacks. Consumers and employees are taught to recognize suspicious emails (phishing), suspicious text messages (smishing), and manipulative phone calls (vishing). Training programs provide a vital line of defense by helping individuals identify red flags and detect subtle inconsistencies such as slight variations in wording, unexpected requests, or small errors in communication.
A well-trained employee might recognize unusual wording in an e-mail supposedly coming from a colleague, or a voice message requesting confidential information that “comes from” an executive who should already have access to it. Consumers can also benefit to some extent if they are trained to avoid mass-produced smishing and vishing scams.
But even the best-trained humans can make mistakes, and stress, fatigue and cognitive overload can impair judgment, making AI attacks more likely to succeed.
Tomorrow will be favored by AI
Over the next 2-3 years, AI attacks will have access to more data and larger and better Large-Scale Language Models (LLMs), allowing them to generate more convincing, context-aware interactions that mimic knowledgeable human behavior with astonishing accuracy.
Today, AI-powered attack tools can create emails and messages that are nearly indistinguishable from those of legitimate contacts. Voice clones can also mimic the speaking voice of virtually anyone. In the future, these techniques will be combined with advanced deep learning models to incorporate vast amounts of real-time data, spyware, and speech patterns into near-perfect deepfakes, making AI-generated attacks indistinguishable from human contact.
Already, AI-based attacks offer the following benefits:
Seamless Personalization: AI algorithms can analyze vast amounts of data to customize attacks to an individual’s habits, preferences, and communication style.
Real-time adaptation: AI systems can adapt in real time, revising their tactics based on the responses they receive: if an initial approach fails, the AI can quickly pivot and try different strategies until it finds an effective attack.
Emotional manipulation: AI can exploit human psychological weaknesses with unprecedented precision: for example, an AI-generated deepfake of a believable family member in distress can evade rational scrutiny and elicit an immediate, emotionally driven response, convincingly calling for urgent help.
At Appdome, we are beginning to see exploits using AI chatbots overlaying attacks on mobile applications to engage in seemingly harmless conversations with customers and employees. Some brands are beginning to prepare for the same attacks executed through AI-powered keyboards that victims have installed on their mobile devices. In each case, the overlay or keyboard can gather information about the victim, persuade the victim, present malicious options, or compromise security, accounts, or transactions on the victim’s behalf. Unlike today, where anomalies can be detected and individuals can control actions, the future of AI-driven attacks will include autonomously created interactions within applications and AI agents acting on behalf of the victim, removing humans from the attack lifecycle entirely.
The Future of Security Awareness Training
As AI technologies evolve, traditional security awareness training faces an existential threat and the room for human error is rapidly disappearing. The future of security awareness training requires a multi-faceted approach that leverages real-time automated intervention, increased cyber transparency, and AI detection, along with human training and intuition.
Technical attack intervention
Security awareness training should expand beyond attacks to teach individuals to recognize true technical interventions by brands and companies. Even if an individual cannot distinguish between real and fake interactions by attackers, it should be easy for them to recognize system-level interventions designed to protect users. Brands and companies can detect when malware, espionage, control, and account takeover technical techniques are being used and use that information to intervene before real damage occurs.
Improving Cyber Transparency
For security awareness training to be successful, organizations must increase cyber transparency so users understand the defensive response expected of their applications and systems. Of course, this first requires that those applications and systems have robust defensive technical measures in place. Still, corporate policies and consumer product release notes should state “what to expect” if a threat is detected by the brand or corporate defenses.
Recognize AI and AI agents that interact with your app
Brands and businesses need to put in place defenses that detect the unique ways machines interact with applications and systems, including typing, tapping, recordings, patterns of movement within an app or on a device, and even the systems used for these interactions. Non-human patterns can be used to trigger alerts to end users, enhance due diligence workflows within applications, or take additional authorization steps to complete a transaction.
Preparing for an AI-powered future
The rise of AI-powered social engineering attacks is a sea change in the cybersecurity landscape. For security awareness training to remain a valuable tool in cyber defense, it must incorporate application and system-level interventions, increased cyber transparency, and awareness of automated interactions with applications and systems. Doing so will help protect brands and businesses from the inevitable rise in AI-powered fraud, enabling a safer future.
About the Author
Tom Tovar is CEO and co-founder of Appdome, the only fully automated, integrated mobile app defense platform. Today, he is a programmer, hacker, and business leader. He began his career as a Stanford-educated corporate and securities lawyer with a focus on technology. He has served in board and C-level leadership roles at several cyber and technology companies, providing practical advice.