In this blog post, we delve into the emerging use of generative AI, including OpenAI's ChatGPT, and the cybercrime tool WormGPT, in Business Email Compromise (BEC) attacks. Highlighting real cases from cybercrime forums, the post dives into the mechanics of these attacks, the inherent risks posed by AI-driven phishing emails, and the unique advantages of generative AI in facilitating such attacks. Contact us here if you’d like to learn more and protect your organization from advanced BEC attacks.
How generative AI is revolutionizing BEC attacks
The progression of artificial intelligence technologies, such as OpenAI's ChatGPT, has introduced a new vector for business email compromise (BEC) attacks. ChatGPT, a sophisticated AI model, generates human-like text based on input. Cybercriminals can use such technology to automate the creation of compelling fake emails, personalized to the recipient, thus increasing the chances of success for the attack.
Consider the first image above, where a recent discussion thread unfolded on a cybercrime forum. In this exchange, a cybercriminal showcased the potential of harnessing generative AI to refine an email that could be used in a phishing or BEC attack. They recommended composing the email in one's native language, translating it, and then feeding it into an interface like ChatGPT to enhance its sophistication and formality. This method introduces a stark implication: attackers, even those lacking fluency in a particular language, are now more capable than ever of fabricating persuasive emails for phishing or BEC attacks.
Moving on to the second image above, we now see an unsettling trend among cybercriminals on forums, evident in discussion threads offering “jailbreaks” for interfaces like ChatGPT. These “jailbreaks” are specialized prompts that are becoming increasingly common. They refer to carefully crafted inputs designed to manipulate interfaces like ChatGPT into generating output that might involve disclosing sensitive information, producing inappropriate content, or executing harmful code. The proliferation of such practices underscores the rising challenges in maintaining AI security in the face of determined cybercriminals.
Finally, in the third image above, malicious actors create custom modules similar to ChatGPT, but easier to use for nefarious purposes. Not only are they making these custom modules, but they are also advertising them to fellow bad actors. This shows how cybersecurity is becoming more challenging due to the increasing complexity and adaptability of these activities in a world shaped by AI.
Uncovering WormGPT: a cybercriminal's arsenal
Our team recently gained access to WormGPT through a prominent online forum often associated with cybercrime. This tool is a blackhat alternative to GPT models, explicitly designed for malicious activities.
WormGPT is an AI module based on the GPTJ language model developed in 2021. It boasts various features, including unlimited character support, chat memory retention, and code formatting capabilities.
As depicted above, WormGPT was allegedly trained on diverse data sources, particularly malware-related data. However, as the tool's author decided, the specific datasets utilized during training remain confidential.
As you can see in the screenshot above, we conducted tests focusing on BEC attacks to assess the potential dangers associated with WormGPT comprehensively. In one experiment, we instructed WormGPT to generate an email to pressure an unsuspecting account manager into paying a fraudulent invoice.
The results were unsettling. WormGPT produced a persuasive and strategically cunning email, showcasing its potential for sophisticated phishing and BEC attacks.
It’s similar to ChatGPT but has no ethical boundaries or limitations. This experiment underscores the significant threat posed by generative AI technologies like WormGPT, even in the hands of novice cybercriminals. Contact us here to protect your organization from BEC attacks like WormGPT and others.
Benefits of using generative AI for BEC attacks
So, what specific advantages does using generative AI confer for BEC attacks?
Exceptional Grammar: Generative AI can create emails with impeccable grammar, making them seem legitimate and reducing the likelihood of being flagged as suspicious.
Lowered Entry Threshold: Generative AI democratizes the execution of sophisticated BEC attacks. Even attackers with limited skills can use this technology, making it accessible to a broader spectrum of cybercriminals.
Ways of safeguarding against AI-driven BEC attacks
In conclusion, AI's growth brings progressive, new attack vectors. Implementing strong preventative measures is crucial. Here are some strategies you can employ:
BEC-Specific Training: Companies should develop extensive, regularly updated training programs to counter BEC attacks, especially those enhanced by AI. Such programs should educate employees on the nature of BEC threats, how AI is used to augment them, and the tactics employed by attackers. This training should also be incorporated as a continuous aspect of employee professional development.
Enhanced Email Verification Measures: To fortify against AI-driven BEC attacks, organizations should enforce stringent email verification processes. These include implementing systems that automatically alert when emails originating outside the organization impersonate internal executives or vendors, and using email systems that flag messages containing specific keywords linked to BEC attacks like "urgent", "sensitive", or "wire transfer." Such measures ensure that potentially malicious emails are thoroughly examined before any action is taken.
Don’t wait for AI-powered BEC attacks to hit your inbox.
Learn how generative AI is transforming the threat landscape and what you can do to stay ahead. Our team can help you identify vulnerabilities, strengthen defenses, and build resilience against advanced phishing and BEC tactics.
Contact us today to protect your organization from the next wave of AI-driven cybercrime.
.png)
