Getting past guardrails on a road? Bad. Getting past guardrails for AI models? Lucrative for AI jailbreakers—provided they know where to sell whatever data they steal.

Open Measures, a threat intelligence resource that allows attacker search across platforms, recently published research on the increased sophistication of AI-infused cybercrime—and where AI jailbreakers and other attackers are congregating. The report shows how vast the jailbreaking ecosystem is, spread across a number of sites, primarily 4chan, Discord, and Telegram.

Report author Ryan Aliapoulios, a researcher with Open Measures, told IT Brew that the space has evolved for cybercriminals.

“AI jailbreaking, at one point, was a niche community interest,” Aliapoulios said. “Now it’s become a cross-platform ecosystem that has its own division of labor, where each social platform has its own role in the ecosystem.”

Where are these mysterious markets?—EH

’Til governance do us part…

Companies are calling it quits on deployed AI-powered customer communication agents, often due to governance issues around data leakage and hallucinations.

According to early findings from AI cloud communication provider Sinch’s AI Production Paradox report, 74% of companies rolled back or cancelled customer communication agents due to a governance failure. Those companies that self-described their guardrails as “fully mature” had an 81% rollback rate; the report’s authors suspect this may be due to increased internal monitoring that allows for better detection of vulnerabilities.

The findings are based on a January survey of 2,527 executives across six industries who are “directly responsible for their organization’s AI communications strategy.”

How bad is bad?—BM

Why wait until an AI system is done being built to test against potential flaws?

Ram Shankar Siva Kumar, a data cowboy (yes, that’s his official title) at Microsoft, leads the team responsible for RAMPART, an open-source tool that red teams and other cybersecurity pros can use to test AI agents’ vulnerabilities.

“The reason why we have tooling investments is it’s really hard to do all these things manually,” Siva Kumar said.

The team is also working on Clarity, another open-source tool that offers feedback to developers writing code.

Running RAMPART. The tool, which is based on top of Microsoft’s PyRIT cybersecurity automation framework, injects red team tools into developer workflows. For example, if a developer is building an AI agent, they can use RAMPART as a tool to simulate prompt injections.

More on automated read teaming.—CN

CollabWORK connects you to the hidden job market through IT Brew and other trusted channels. Browse roles curated specifically for this community by clicking through to the job board.