What the latest alliance for open source AI means for cybersecurity

Like Voltron, some of the biggest names in tech are deciding the best way to secure their infrastructure against AI-powered attacks is to bond together into a giant robot open-source alliance.

On Monday, Nvidia announced the Open Secure AI Alliance. In addition to Nvidia, membership includes IBM, Hugging Face, Palantir, OpenClaw, ServiceNow, Red Hat, and others. The goal: ensure that cybersecurity pros have open-source tools and techniques that they can use to secure infrastructure and find vulnerabilities, particularly as malicious AI expands the collective attack surface.

This announcement follows an open letter from some members of the alliance, alongside others, regarding concerns around potential bans on open-weight models.

Justin Boitano, VP of enterprise AI at Nvidia, told IT Brew that open development can help keep AI safe.

“The world is a safer place when there’s open development happening through this technology stack,” Boitano said. “It produces stronger defenses across many different companies, allowing independent researchers to kind of review people’s solutions, identify potential blind spots, kind of question each other’s assumptions, and make sure that we just don’t have blind spots as an industry.”

What contributing looks like. Boitano said the alliance wants to support the open-source community through sharing new tools and innovations, ultimately aiming to “democratize access to agentic defense.”

Boitano pointed to the recent incident in which advanced OpenAI models hacked Hugging Face’s infrastructure as an example of why it’s critical for cybersecurity pros to have the capability to observe and respond to agentic AI attacks.

Where it’s all going. Nvidia, the Linux Foundation, and CrowdStrike are some of the alliance members that are also a part of Project Glasswing, the initiative to test Claude’s advanced Mythos model against cybersecurity tactics and infrastructure.

Boitano said that when Project Glasswing first started, NvidiaV’s cyber team saw it as a “great opportunity” to rethink agentic security.

Frank Dickson, IDC’s group VP for security and trust research practice, told IT Brew it’s crucial to enlist leaders to address security concerns around open source and ensure the technology remains an enabler.

“We’ve already seen a lot of times where bad people will contribute malicious things to open source, and now distributing problems at scale or maliciousness at scale,” Dickson said. “Having a shared vested interest to make sure open source is valid is where this is going.”

Dickson said this open-source alliance is the next extension of what Project Glasswing was trying to do. But in contrast to Glasswing, which was evaluating a model that could have posed an immediate threat, this alliance is focused on building a safe foundation for the future.

Dickson added that he expects to see more efforts to start signing code, including open-source code, which means attaching a verifiable cryptographic signature to a piece of software to show that the code had not been tampered with. Such moves contribute to the trustworthiness of an open source product for organizations.

“The whole model is a little bit fraught, because open source is contributed, open source is openly managed,” Dickson said. “It’s a bunch of people that are doing things for free, and so how do you validate that?”