Companies kick the tires on open-weight models amid AI cost crunches

For the last year or so, Fitz Nowlan watched with interest as Chinese open-weight models narrowed the performance gap against those offered by big-name labs.

The VP of AI and architecture for IT software company SmartBear, Nowlan said there are big potential draws to these free-to-download models, like Z.ai’s GLM-5 or DeepSeek v4: cost-efficiency, lower latency, and avoiding the risk that a lab could create a competing software product.

Now he’s undertaking an extensive experiment to see where these models might make financial sense.

“We’re in the throes of that investigation now,” Nowlan said. “If we have the cost-benefit in our favor, we can start routing some of those prompts to the open-weight model.”

Nowlan isn’t alone. With CIOs and CTOs wringing their hands over crushing AI bills and more cost-efficient Chinese models starting to rival some of the capabilities of expensive frontier lab offerings, it’s no surprise that some businesses have become open-curious.

Open-weight models are far from the norm in enterprise AI right now, outside of some niche corners (they’re more popular in highly regulated industries). But several execs we spoke with said they’re in the midst of experimenting, especially as multi-model strategies become more popular.

“It’s definitely a top topic with clients,” PwC Global Chief AI Engineer Scott Likens told us. “I don’t see it at scale at this point—there might be specific areas that they’re doing some experimentation—but it’s probably one of the top three conversations, obviously caused by the token curve that went straight up.”

This wave of interest comes amid a seemingly eternal debate about whether the future of generative AI should be built around the tightly controlled proprietary models that currently dominate the industry, like ChatGPT, Gemini, and Claude, or open models that make their parameters, or weights, freely available to download and modify.

Since Meta sidelined Llama, the most popular open-weight models have come from Chinese AI companies like DeepSeek and Moonshot. But US companies offer them too, including Nvidia, OpenAI (gpt-oss), and Google (Gemma). Earlier this month, Meta CEO Mark Zuckerberg signaled a shift back toward open-weights with the release of Muse Glimmer and a manifesto about the importance of openness.

Up-front costs

While open models may be cheaper in the long run, they can be pricey for businesses to set up initially, whether on the cloud or using an on-premise server. Harman Kaur, CTO of cybersecurity and systems management company Tanium, said there are a lot of decisions to be made in the beginning—which model to use, how to host it, which use cases to start with.

Rigorous evaluations and potential fine-tuning or other customization are needed, since the training datasets of open-weight (versus fully open-source) models are still a black box, and extra diligence should be done around sourcing to avoid malware attacks.

Kaur is currently evaluating the Inkling model from Thinking Machines Lab, a buzzy startup from former OpenAI CTO Mira Murati. Tanium has found the performance of some tasks to be on par with closed models, but there are other pressing questions, like what kind of governance scheme is needed.

Part of that question is whether to source Chinese open-weight models, which “tend to perform better,” she said. “But is that something that we want to provide to our customers? I don’t know.”

SmartBear’s Nowlan said it’s important to be mindful that AI models can carry biases instilled by their creators, such as companies operating under the restrictive Chinese government. But that’s not much of a concern for the technical tasks that SmartBear is testing them for.

“Tool calling, like routing through these workflows, probably wouldn’t be too tainted, for example, by political beliefs,” Nowlan said.

History repeating

Ara Kharazian, lead economist at the expense platform Ramp, said business spending on open-weight models has been growing at a “marginal but meaningful” rate, but not enough to make Chinese upstarts much of a threat to frontier lab dominance.

“Adoption of open-weight models and Chinese models in general has spiked when there has been some hype cycle around the hot new Chinese model coming out,” Kharazian said.

The first cycle came after an ultra-cheap early DeepSeek model led investors to dump AI and chip stocks early last year out of fear that frontier models could be replicated more efficiently. More recently, Moonshot’s Kimi K3 grabbed headlines for its benchmark scores against top models.

“I think [open-weight adoption] will stay pretty cyclical. Some of it will go back to policy, if there’s some regulation,” Likens said. “I don’t think I see [competition between open and closed] stopping in the near term.”

Iz Beltagy, director of AI research at the Allen Institute for AI, told us there are two types of businesses that tend to be most interested in open models. First, those in heavily regulated industries, like financial services or healthcare, where locally hosting data is important. Then there are businesses that have passed the initial AI prototyping phase and are now concerned with optimizing for costs at scale.

“Once you have a decent amount of traffic, once you have data from your users and interactions with users, once the unit economics matter and you want to optimize your revenue, then it becomes a no-brainer for you to consider,” Beltagy said.

SmartBear fits that latter category, according to Nowlan. The company spent the past year and a half less concerned with costs in order to see what’s possible. But now that his team has found valuable use cases, they’re looking to cost-optimize.

“We now feel like we’re getting to the point where we have some value we want to protect,” Nowlan said. “We intend to be selling this value for years into the future. So now let’s ask ourselves: Could we deliver this value for less?”

Rajeev Singh, CEO of work management platform Smartsheet, said the company uses mostly Anthropic and Amazon models right now, but is also exploring some open-weight models. Singh said most software vendors are likely considering them as a cost hedge against the volatility of token prices from major frontier providers.

Maybe in the future, Smartsheet might route simple tasks to open-weight models, Singh said. But right now, it’s favoring frontier models despite the higher costs.

“Today, we’re biasing toward speed. We’ve gotta get to the market. We gotta deliver to our customers,” Singh said. “As the transactional volume goes up, we’ll start to optimize for cost.”