What is LLMjacking, and why should IT pros care?

Sponsored by

Your AI strategy has a security problem: 43% of IT leaders now cite security concerns as the top barrier to expanding AI agent deployment, according to research from JumpCloud. Before giving agents more access to apps, data, and workflows, make sure you can govern the identities behind them. See how to secure every identity, human or not.

As anybody who follows cybersecurity knows, when a new technology emerges, it’s usually followed by a threat actor trying to -jack it up.

There’s clickjacking (tricking someone into hitting a disguised URL), sessionjacking (stealing a token to impersonate a user and gain their web access), and DNSjacking (redirecting someone to an attacker-controlled destination). And now, with attackers trying to take over large language models, we have…LLMjacking.

This is more than just freeloadin’ freecoders using compromised access to steal AI services for their personal use: LLMjackers are building attack tools now, too.

LLMjacked up. Cloud security company Sysdig (which claims to have coined “LLMjacking” in May 2024) describes the tactic as “using stolen cloud credentials to gain access to a victim’s paid AI model services and leverage the compute power.”

Crystal Morin, senior cybersecurity strategist at Sysdig, has seen an evolution in how attackers leverage cloud-based LLMs once they obtain access. In 2024, Sysdig observed threat actors employing stolen credentials to gain free access to victims’ AI services for free; later, these attackers spun up new models, then pivoted to underground marketplaces for LLMjacking.

In a June blog post, Sysdig noted threat actors using an internet-exposed Ollama model to power an automated attack tool. For Morin, that represents a clear change in purpose, suggesting attackers want to do more than use an LLMjacked model to get a quick answer to a burning question or write an essay.

“To see [LLMkjacking] go from using someone else’s LLM to do college homework, to using somebody else’s LLM to create an offensive tool, in just a matter of about 24 months, I think is quite a large shift in the evolution,” Morin told IT Brew.

Sound familiar? Dan Moore, senior director of CIAM strategy and identity standards at customer identity and access management platform FusionAuth, sees the LLMjacking situation as similar to, another “jack,” cryptojacking, the compute theft used for cryptocurrency mining about a decade ago.

To help thwart LLMjacking, Moore recommends short-lived credentials, not static API keys, as well as enforcing least privilege. “You get access to the specific model that you know you need, and maybe you time-bound that,” he said. (Here is guidance from Anthropic, OpenAI, and AWS on time-bound keys, to demonstrate three examples.)

Also, the prospect of an LLMjack requires a close look at usage across an organization—and price and activity limits may be necessary, according to Morin.

“If the attacker isn’t automating an extremely high amount of token usage, they might fly under the radar, and you’re not going to see it. But if they’re using 2 billion tokens a day to automate some excessive project that they’re working on, then you may see a spike,” Morin said.

Morin recommends following authentication practices, including rotating credentials and restricting who can access and stand up AI infrastructure. (OpenAI this April announced that members of its Trusted Access for Cyber program—practitioners accessing the company’s “most cyber capable and permissive models”—must switch to passkeys or physical security keys.)

The Ollama instance could be found on the open internet, according to Sysdig’s blog post, no authentication needed. With unauthenticated model servers, threat actors don’t have to worry about LLM services jacking up the price.

“Fortunately, we did not see that offensive tool get used against another third party, but it was being tested. It was definitely capable of being used against any kind of vulnerable victim environment,” Morin said. “We expect to see more LLMjacking because these [AI models] are expensive.”