Are open-source and open-weight AI models worth the risk?

For some companies, cheaper might be better when it comes to AI—as long as the right security measures are in place.

Organizations navigating AI implementation are already turning away from tokenmaxxing and trying to restrain their operating costs, especially with cutting-edge “frontier” models produced by Anthropic, OpenAI, and other AI giants.

That creates an opening for Chinese tech companies like Moonshot AI and Z.ai (formerly Knowledge Atlas Technology Joint Stock Co.), which are offering open-weight and open-source alternatives that organizations can theoretically deploy at cheaper cost than the frontier models.

With open-weight AI models, the “weights” that govern the model’s interpretations of inputted data and how it responds are released to the public for downloading and customization. Open-source AI models, meanwhile, allow users to download and tweak everything, including source code and training data. Provided a company has the infrastructure to run these models, they can prove a less costly alternative to the companies charging per-token for AI usage.

While open-weight and open-source models have traditionally been viewed as less powerful than frontier models, things may be changing. For example, Moonshot AI claims that its latest model Kimi K3 can compete on the capabilities front with US-made models.

But organizations also have security concerns around the use of open-source and open-weight models, especially because attackers can launch sophisticated attacks against AI systems.

Rush job. Victor Wieczorek, SVP of offensive security at GuidePoint Security, told IT Brew that bad actors can inject malicious code into repositories for open-source and open-weight models, “poisoning” those models into altering their behavior or exposing vulnerabilities. Researchers and IT professionals “feeling the crunch” to implement AI might not do their due diligence of the model’s codebase until it’s too late.

“Because there’s now a desperate effort to either race to get this access to these technologies or to ensure that they are always going to have the influence that they want, it’s generating a lot of bad behavior of people moving too fast, not fully inspecting or thinking about the models or the harnesses, or artifacts they are pulling into their environments,” Wieczorek said.

Specifically, Wieczorek recommended that IT professionals employ harnesses to allow open-weight models to operate securely through code execution within a sandbox or calling an API. (According to Databricks, the harness evaluates the potential solutions within an isolated environment to test the solutions and try something else if the tests fail.)

Decisions, decisions. Nonetheless, Wieczorek said that open-weight models are going to remain in the picture for many organizations, especially since they can help prevent vendor lock-in. IT teams can also use frontier models to tune localized, open-weight models for specific uses.

“As the compute starts to become limited, then you’re going to see engineers do what engineers do best,” Wieczorek said. “They can turn around and use local models or open-weight models and guide them and tune them.”

Even with the potential risks, IT teams need to consider open-weight and open-source models in the context of their budgets and needs.

“The financial aspect of models is not how it is going to be in a year or three years, five years from now, and you need to take that into consideration as well,” said Joanna Chen, CISO for Dashlane. “There have been a lot of security researchers that have had a lot of success with open-weight models, they might not be as good, but they may surpass the proprietary models in the near future.”