What is workflow identity hijacking?

Sponsored by

Your AI strategy has a security problem: 43% of IT leaders now cite security concerns as the top barrier to expanding AI agent deployment, according to research from JumpCloud. Before giving agents more access to apps, data, and workflows, make sure you can govern the identities behind them. See how to secure every identity, human or not.

Closed mouths don’t get fed—but open ones do, regardless if they are malicious or not.

A Noma Security researcher has discovered an AI workflow attack vector that enables cybercriminals to obtain sensitive information just by asking for it.

How it works. The vector, which was detailed in a Sept. 9 Noma Labs blog post, is known as “workflow identity hijacking,” and occurs when malicious actors send seemingly ordinary requests for sensitive information to unauthenticated entry points (think web forms or support inboxes) that generate a response via AI. In a workflow identity hijacking, an attacker, for example, may use an organization’s public support email to request and receive sensitive information from a finance director’s most recent email.

Sasi Levi, security research lead at Noma Labs, told IT Brew the security gap is a result of the AI workflow using privileges the attacker does not possess.

“There are a lot of identity problem issues here because the task is doing what it needs to do, but it doesn’t know that the attacker is the persona that runs this and not the developer,” Levi said. “And this is a very big issue.”

Prompt season. IT Brew has previously reported on prompt injections, an AI attack vector where malicious actors try to deceive LLMs into sharing sensitive information or bypassing policies and instructions. Levi, in his blog post, said workflow identity hijackings differ from prompt injections because the attacker is exploiting two different things.

“Prompt injection manipulates how the model follows instructions, while workflow identity hijacking exploits whose authority the workflow uses when executing a valid request,” he wrote.

Prompt injections and workflow identity hijacking aren’t the only threats enterprises need to watch out for. In July, IT Brew reported on the rise of token torching, which occurs when attackers use malicious prompts to drain a company of its AI tokens.

How to protect against workflow identity hijacks. In his blog post, Levi said Noma Labs identified and reported a workflow identity hijacking risk vector within Google Workflows. According to Noma Labs, Google has since acknowledged its report and confirmed a fix.

To mitigate the risk of workflow identity hijacking, Levi said organizations should make sure that their AI workflows verify the domain of a request before acting on it.

“Basically, I’m telling the first task of the workflow, ‘Please check who sent the email, and if you see that the email doesn’t come from our organization, please drop it,’” he said.

But that’s not all. Levi, in an email, added that companies should treat LLM outputs as untrusted inputs, and should include an “explicit authorization checkpoint between what the model produces and any sensitive action the workflow performs.”

“Finally, organizations need to think about their entire workflow architecture, not just whether the model itself is secure. Model guardrails should never be or become your authorization layer.”