AI risks are getting more sophisticated: report

IT pros, freeze! You have the right to remain silent. The very AI tools intended to help ease your workflows can and will very likely be used against you.

That’s according to Cisco’s second annual State of AI Security report, which found the vulnerabilities it analyzed last year have quickly matured into more pressing risks. Amy Chang, a lead contributor on the report and leader of AI threat and security research at Cisco, told IT Brew that malicious actors are using AI tools to their advantage as companies continue to figure out how to leverage the technology, often with security as an afterthought in the process.

“Attackers and the threat actors are similarly experiencing that momentum gain and are able to utilize existing [AI] tools that are for productivity…and then using them for malicious end goals,” Chang said.

Supply chain pain. The report identified several blind spots and attack vectors that could be leveraged against AI-enabled companies, including an increasingly fragile supply chain. Last year, a UK research paper funded by Anthropic concluded that just 250 “poisoned” documents injected into training data can compromise the safety of an LLM regardless of its size.

But that’s not all. According to the report, most model repositories “provide no cryptographic assurance of who trained a model, what data influenced it, whether it has been modified since publication, or even its country of origin.” As a result, malicious activity like backdoor insertions, a type of attack that enables attackers to get unauthorized access to compromised systems—can fly under the radar. Chang said the industry needs to take a “unified approach” to addressing this issue.

“It is incumbent upon the model hosting organizations like Hugging Face to be able to institute security checks to the extent that they’re able,” Chang said, adding that professionals and companies downloading from these platforms have their own responsibility to ensure the models are safe.

Excessive, much? Prompt injections and jailbreaking have also evolved into more sophisticated threats, according to the Cisco report, which concluded that current GenAI models, though more sophisticated, are still at risk for such threats, especially when done through multiple-step attacks.

“Because of the nuance of human language, you are able to then obfuscate the malicious intentions,” Chang said. Both adversaries and researchers are also sharpening their ability to retrieve unauthorized information from LLMs. IT Brew previously reported on one threat intelligence researcher’s success in sending malicious requests to several LLM models by disguising it as a fictional story.

Attack of the agents. Cisco’s report claims the theoretical risks associated with AI agents are now becoming a reality. Malicious actors are not only employing agents to help carry out their nefarious deeds, “allowing them to operate at a scale and speed that human teams cannot match,” but also exploiting how they communicate with one another for denial-of-service and agent impersonation attacks.

Well…now what? With all this in mind, Chang said it’s important for IT professionals to stay up-to-date with AI threats and vulnerabilities beyond the more mainstream risks: “If you take these things into your tech stack or your enterprise environment without fully understanding where the risk could potentially lay, that creates a lot of opportunity for the attack surface to expand without fully understanding.”