Is AI giving companies a tech debt problem?

AI has generated a lot of hype within the IT industry. But the excitement may be creating a financial bubble—and a lot of tech debt as companies try to integrate AI into their IT infrastructure.

If that wasn’t enough, there’s also a timing issue. Vineet Jain, the CEO and co-founder of AI-powered cloud content management and data security platform Egnyte, said that while companies are investing heavily in AI, they’re unsure of when demand for those AI-powered services will actually appear.

“While the hype cycle on AI is probably at its zenith, we are still in a huge state of flux where, from a consumer or the customer perspective, there’s a lot of clamoring and excitement about AI—particularly generative AI,” Jain said. “You’re seeing that now there is a surge of…public or private debt that these companies are wading into to continue fueling their investments, and that is causing a lot of concerns.”

Companies of all sizes. Blake Crawford, the founding partner and chief technology officer for Fusion Collective, said that many mid-size clients are racking up technical debt as part of their AI implementation.

“[Mid-size companies] have enough capital to have rapidly developed some very interesting AI pilots and unfortunately done so in very questionable and inefficient ways,” Crawford said. “But it’s not necessarily the fault of the organization, it’s more the way the AI industry has established itself and the speed with which it’s been moving.”

Crawford described one client who built a pilot for an LLM product that used a two-stage query to surface vital information for users. The client faced a significant scale issue: if the model was deployed across a 500- or even 1000-person user base, “things just get completely out of control, the budgets go crazy.”

“Speed starts to deteriorate, nothing really works anymore as they had intended,” Crawford added. “A lot of that just has to do with the speed with which all of this has come to be; not necessarily the inherent fault of the organizations or the developers themselves, it’s just the tools that they’re using aren’t quite mature enough to scale to the level that they want for it to scale.”

Is it all bad? David Spreng, the CEO of Runway Growth Capital and former venture capitalist, told IT Brew that the financial debt issued to technology companies in the last couple of years isn’t concerning, but “actually quite positive.”

“The amount of debt [Big Tech companies are] taking on is not scary, in an absolute number it’s a lot but relative to the size of their balance sheets, it’s insignificant,” Spreng said. “When you start to get to upstarts…the amount of debt they’ve taken on, relative to their overall capitalization, I think that’s more concerning.”

Experts within the industry agree that the market is in a heavily subsidized era of AI at the moment, according to Kevin Novak, a managing partner at Rackhouse Venture Capital, which is causing anxiety for what things might look like years from now.

“If the bubble pops, I think what ends up happening is just AI starts to cost quite a bit more,” Novak said. “I think, unfortunately, because you’re layering on multiple layers of subsidies in the sense of, right now the LLMs are cheap, the workflows and broad, horizontal applications are cheap…But effectively, they’re going to start to have to pay more as those lower layers have to rationalize.”

But…is there a bubble? Reuters has reported on the rising debt that’s bankrolling AI investments. Investors are concerned about a “bubble” of AI spending that could pop if revenues don’t match expenditures, according to Yahoo! Finance.

Jain doesn’t think there’s a bubble—instead, he describes the exuberant AI spending as a soufflé “going up and up”—which will eventually settle and leak a bit.

“Everything will not go into smithereens, there will be a rationalization,” Jain said.

In the meantime, Jain said the market will see a “huge frenzy” of AI investment in the next 18 to 24 months.

“A lot of smaller startups will go away, even some of the bigger companies may not completely go away but shrink quite a bit because there only can be so many players in the space,” Jain said. “This thing is here to stay, but [the] next two years, you will see a lot of changes, a lot of mutations, a lot of destruction felt before it starts picking up again.”

Spreng said that, with the uncertainty around an AI bubble, professionals need to act, starting with raising “as much money as you possibly can now.” Companies should also consider what a “soft landing” might look like.

“What will happen is there’ll be some kind of bubble, and it will begin in the public market and public stocks will massively correct and then [venture capitalists] will become less interested and will start not living up to their promises of further investments,” Spreng said. “Companies will find themselves somewhere halfway to the end-zone of whatever plan they built.”