‘HashJack’ demo hides malicious instructions in URL

When it comes to URLs, if you leave a message after the #, a hacker just might get back to you with a scam.

A recent demo from IT security company Cato Networks showed how placing malicious instructions after the hashtag in a lengthy, otherwise legitimate URL can fool an AI browser’s large language model into obeying the commands.

While Microsoft and Perplexity reportedly fixed the “HashJack” vulnerability in their browser offerings after seeing the tactic in action, new prompt injection ideas keep appearing, threatening the security of emerging tech like AI browsers.

“One of the major vulnerabilities for AI systems is prompt injection,” Cato Senior Security Researcher Vitaly Simonovich told IT Brew, referring to a technique wherein an attacker inputs text that tricks a large language model (LLM) into following potentially malicious instructions.

How it works. Simonovich, having previously tricked large language models with long stories, decided to try a long URL.

The security pro embedded malicious directives into the URL. When some chatbot-equipped AI browsers load the page, he found, the bot pulls in the URL as context for a user query. Hidden commands in the address are then fed into the large language model, and in some cases, the LLM followed those commands. Because the URL fragments stay within the browser, the demonstrated attack potentially evades traditional network-level detections.

The Cato Networks post demonstrated this technique in several ways:

• A prompt in Google’s Gemini asking, “What are the new services and benefits?” led to an execution of a callback phishing scam.
• A loan question posed to Perplexity’s AI assistant Comet hid instructions to send a user’s banking data to a threat-controlled URL.
• A “new services” query led to Microsoft's Copilot displaying a fraudulent “verify your account now” login option.

Although Microsoft and Perplexity applied fixes to the prompt injections, according to Cato’s blog, Google’s “issue remains unresolved at time of writing.” (Google did not respond to IT Brew’s request for comment by publication.)

Prompts aplenty. Researchers have been demonstrating new prompt injections somewhat regularly these days—one recent report even revealed how a “poetic” structure in a query can force an AI browser to break down.

“The LLMs are evolving, just like web applications are constantly evolving. There’s always a new version being released. With new versions and new technology come new vulnerabilities and new human ingenuity,” prompt-injection researcher Joey Melo told IT Brew in August.

A day after OpenAI released its ChatGPT Atlas browser on Oct. 21, the company’s CISO wrote on X that prompt injection was “an emerging risk” being thoughtfully researched and mitigated.

“Our long-term goal is that you should be able to trust [a] ChatGPT agent to use your browser, the same way you’d trust your most competent, trustworthy, and security-aware colleague or friend,” Dane Stuckey wrote at the time.