How IT pros should think of AI security risk

AI risk is all over the news as the public tries to understand the threat that models and agents pose. But for IT teams, the real question is how much the danger changes their security posture, and how to communicate that change to their organizations.

Recent, well-publicized breaches like the Hugging Face attack have highlighted the importance of managing the risk of increasingly capable AI agents. Whether that marks a new risk or an increase in a known one remains an open question.

Changing environment. Ramy Rahman, senior solutions engineer at ArmorCode, told IT Brew that he sees the potential for damage as real and worth planning for. The new agentic swarms are fundamentally different from older bots that lacked the capabilities of modern AI.

Communicating that change in power and capability outside of the IT team, though, can be daunting.

“We can make bots that scrape your website and harbor data and cache images,” Rahman said. “But now, we’re talking about agentic workflows or automated agents, and the difference between those types of verbiages is what is a difficult thing to explain; if you talk about something that has an agentic workflow or agentic AI, you now have to redefine it.”

Viewpoints matter. A correct assessment of the threat is important. That’s how Aya Ibrahim, senior fellow for economic and national security at the AI Now Institute, sees the future of response in a new era of agentic danger. Ibrahim rejects the idea of “rogue or autonomous models,” she told IT Brew, because AI simply does what it’s programmed to do.

“Just because you can’t predict the exact sequence or all of the means by which they’re going to leverage those capacities in service of the tasks that you assign them, or the task that they’re given is imprecise or poorly worded, or is deployed in a setting that doesn’t have the kinds of safeguards and controls that you would otherwise have…that does not mean that they are rogue,” Ibrahim said.

Proper awareness is also front of mind for Santa Clara University Professor Ram Bala, who told IT Brew that due to “the probabilistic nature of these systems” there might be unintended consequences requiring security awareness that adapts to a new reality.

“These are very different computing systems with very different capabilities,” Bala said. “We need to have a very different lens on security.”

Who’s in control? For all its power and capabilities, AI is still subject to governance (or lack thereof), which makes humans working with the technology the real “threat”: responsibility lies in those in charge.

“Do not assume that these companies are acting responsibly,” Ibrahim said. “If you assume that, that means you have to double down on your own sense of guarding your own systems—being proactive about identifying potential risk factors and threats.”

For IT pros looking to implement changes in security posture—or just enforcing existing rules and regulations—it’s critical to remember that agents maywill introduce new forms of risk into IT systems similar to existing human risks, including verification and code vulnerabilities. As Bala put it, the key is to utilize AI in detection so it can catch negative actions and behaviors and resolve them.

“How do I build my security verification, all of those systems around that idea?” Bala said. “That is a concept that I think most IT folks can actually internalize. I think that’ll be useful.”