More than one-quarter of all AI spend is wasted: report

There are good surprises, bad surprises…and then there’s AI surprises.

The lattermost is all too common, according to a new report from Harness, which found that almost three-fourths (72%) of companies experienced a surprise bill or an unexpected spike in AI costs within the past year. (The findings were based on a survey of 700 global FinOps and engineering leaders.)

Unexpected costs aren’t the only thing emptying enterprises’ pockets. Harness found that a little over one-quarter (26%) of overall AI spend is wasted. To put that into perspective, that’s about $130,000 down the drain for companies with a monthly AI budget of $500,000.

The visibility issue. In its report, Harness said surprise AI bills and wasted spend are byproducts of three structural issues within companies. One is gaps in visibility, which Harness’ FinOps Advisory Director Patrick Brogan told IT Brew is partly driven by provider sprawl as organizations tap multiple AI providers at once for services.

“[Providers] have different telemetry, different billing metrics, different ways of sort of valuing tokens, etcetera,” Brogan said. “And it makes it very hard to look at each of those invoices in the silo of one invoice and understand what you’re looking at and not just understand the bill, but trying to understand what the value you as an organization receive by burning tokens, by deploying AI agents, etcetera, into your organization.”

Visibility gaps also exist at the engineer level. Only 45% of engineers understand the cost of what they build.

Whose AI is it anyway? Ownership over AI costs is another area where companies continue to struggle. More than half (52%) of respondents said there isn’t a clear AI cost owner within their organization.

Brogan suggested that unclear authority over AI costs within organizations spawns from having different drivers and influencers of those costs.

“Those platform teams might deploy these services thinking that, ‘Okay, finance has this under control because ultimately they’re the ones who cut the check to the AI providers,’” Brogan said. “Finance might think, ‘Okay, engineers have this under control and certainly know what this stuff costs because they’re the ones who are deploying these services every day.’”

Lax enforcement. Governance gaps are the final problem area identified by Harness. While 73% of companies claim to have an AI cost policy, only 47% admit to fully enforcing it.

A lack of visibility is the culprit behind this gap, said Brogan, who compared the problem to a car that doesn’t tell the driver how fast it’s going: “If my car doesn’t tell me that I’m driving 55 miles an hour, and I see that the speed limit is 35…then that governance policy means nothing to me.”

What’s an IT pro to do? Brogan said IT pros at enterprises where these problems sound all too familiar should focus on assigning an owner to AI costs, which should also help address visibility issues.

“They have to make a conscious and well-communicated decision internally of who’s going to be responsible for overall visibility, and making sure that it’s communicated to each of those contributing teams what their portion of the bill is, and how the decisions they make can influence the bill going up or down,” he said.

As for companies with enforcement gaps, Brogan said they should make sure there’s a “common set of guidelines and governance policies that everybody understands.” He added these policies should evolve with AI over time.

“It’s not a ‘set it and forget it’ type of mentality,” Brogan said. “It’s a constant iteration based on a feedback loop by collaboration with your engineers, your business partners, and the other teams, again, who have a responsibility for the AI bill.”