Phishers may already be using AI to improve their attacks, Darktrace report finds
Darktrace observed a 135% surge in malicious email campaigns with linguistic anomalies since the start of 2023.
Getty Images
Darktrace observed a 135% surge in malicious email campaigns with linguistic anomalies since the start of 2023.
Getty Images
Researchers with security firm Darktrace said they have observed a huge spike in the number of novel social engineering attacks via email since the start of 2023—a trend that could be linked to the relative ease of writing convincing phishing and scam emails with generative AI.
Darktrace chief product officer Max Heinemeyer told IT Brew that across the company’s over 3,000 customers, it had detected a 135% increase in malicious cyber campaigns via email that showed significant linguistic deviation “in syntax, in semantics, in grammar and sentence structure.” He said while the firm could not demonstrate causation or correlation and doesn’t want to fuel AI hype, the time period lines up with widespread commercial availability of tools like OpenAI’s ChatGPT.
“That means less spelling mistakes, longer sentences, better punctuation, more interesting topics, trying to incite people to do something,” Heinemeyer said, adding the attacks were clearly more sophisticated. “There is much more variety in the social engineering we’re seeing come in.”
There’s already evidence that recent generative AI products are finding widespread use in scams and fraud. For example, the Washington Post reported cybercriminals are already using voice cloning to try and trick family members of a target into wiring cash for bail or other emergency legal fees.
Darktrace also commissioned a Censuswide poll of 6,711 employees across the US, Europe, and Australia in March 2023. It found around 70% of respondents said they had noticed a rise in the frequency of scam emails and texts in the past six months. One troubling finding was that the qualities that employees said they viewed as red flags for malicious messages may be ones which generative AI is particularly adept at improving on:
Generative AI may, for example, be able to better frame unusual requests within a context that makes them seem less unexpected, or better pose as an individual belonging to a specific organization or industry. Existing models are also quite good at grammar.
“The most dangerous phishing emails are the ones that are bespoke and well-crafted and tailored to the recipient,” Heinemeyer told IT Brew. “Doing that at scale needs an attacker to spend a lot of time—they need to research the victims, understand what they do.”
“Instead of sitting down and doing all of that research yourself, and then coming up with topics, you could just scrape a victim’s social media profile, or take a Newswire from a company, and then ask a generative AI model to create a believable email based off that, that’s angled towards inciting clicking on a link,” he added.—TM
