Telling the difference between an attacker and an internal AI agent

Sponsored by

Your AI strategy has a security problem: 43% of IT leaders now cite security concerns as the top barrier to expanding AI agent deployment, according to research from JumpCloud. Before giving agents more access to apps, data, and workflows, make sure you can govern the identities behind them. See how to secure every identity, human or not.

Every cybersecurity pro dreads that 3 am ping: someone—or something—is trying to access your organization’s most sensitive systems.

These days, there’s every possibility that the entity attempting access is an internal AI agent trying to complete a legitimate workflow task. But it could also be an automated cyberattack, such as a malicious agent trying to steal critical data.

Paul Reid, VP of adversary research at AttackIQ, suggested that security professionals trying to tell the difference between legitimate agents and their rogue counterparts should start by understanding what “normal” looks like when it comes to agent behaviors on their networks, including tools and data access.

Like other types of software, Reid said, the builders behind agentic AI often prioritize functionality over security. In addition, many organizations lack the telemetry and tooling to monitor how data is flowing through an organization, making it more difficult to monitor agent actions.

“Where humans moved at a certain speed, we were very precise in what we were doing,” Reid said. “With agentic stuff, it’s just overwhelming speed and force that they’re using.”

Insider threat. Jon Baker, VP of threat-informed defense at AttackIQ, said an external attacker trying to gain access to systems via an AI agent might not look that different, behavior-wise, from an organizational agent pushing against its guardrails.

In light of that, it’s critical to ensure that internal agents are given the correct permissions and identity. “What we would be looking for in that situation is behaviors outside of the norm,” Baker said. “That then means all of the usual insider things, are they accessing resources that are unusual? Are they trying to conduct operations that don’t make sense given their role within the organization?”

Brian Golumbeck, managing director of the AI Center of Excellence at Optiv, told IT Brew that enterprises are struggling with governing AI agents.

“This agent should only do certain things, and it has this scope, and therefore if it does something or tries to do something out of that scope, you should kill it,” Golumbeck said. “You should not allow it, or you should request arbitration or a decision on whether that should be permitted.”

Spot the difference. Baker said professionals should look at what happened before and after any attempted access as a way to separate legitimate agent use from malicious.

From there, defenders should try to identify malicious use of gateways for lateral movement through a system.

“You can only achieve B and C if you have first achieved A, so a lot of times that looks like techniques around persistence, or privilege escalation, or lateral movement,” Baker said. Identifying these key choke-points will allow professionals to “prevent access to a whole swath of known behavior.”