Agentic AI is raising more challenges—and opportunities—for tech infrastructure

Keep that engine running—but make sure it’s running smoothly.

For organizations looking to deploy AI solutions across the enterprise—especially agents—relying on complex, hybrid infrastructure is increasingly important. Those at the forefront of AI usage will also need new, powerful solutions to maintain their processing edge.

That’s part of what Robert Blumofe, EVP and CTO at Akamai Technologies, argues in a recent essay, in which he notes how the infrastructure changes necessary to handle agentic AI’s added demands are part of a historical pattern. Blumofe told IT Brew that the cloud is a good example of how technological infrastructure is adapting to the new needs of AI.

“The cloud is pretty central to almost all infrastructure—whether it’s web or internal or other types of applications—and I don’t think AI is an exception to that,” Blumofe said. “Which isn’t to say that there aren’t on-prem use cases; it’s not like the cloud took over 100% of all enterprise compute, but it took over a lot, and I think we see the same for AI.”

What we need. For AI agents, there’s a need to expand capacity for latency and bandwidth; that means an increase in need for creative solutions, given restrictions on infrastructure. As Blumofe put it, the need for more complex operations will make it so the “centralized, high latency, low-bandwidth approach” of on-prem “won’t work.”

But as IT Brew reported earlier in July, there’s a continuing debate within the tech industry over on-prem versus cloud, and whether hybrid solutions (i.e., cloud for the majority of AI use cases, with specialized ones run on on-prem infrastructure) are the way forward. Whatever the industry decides, capital expenditure on the nuts and bolts of the technology is a major concern, as Anil Nanduri, Intel’s VP of AI product management, told IT Brew: “When you start deploying more AI, you are going to have infrastructure that is needed that actually turns into different compute systems, like you will need more CPUs now to execute what the AI is generating.”

System upgrades. CPUs are part of the story, but it’s GPUs, the powerful processors at the heart of AI infrastructure, that Blumofe and others believe will be important going forward. Enterprises should utilize them wisely, Blumofe told IT Brew, with an eye on how to determine where to deploy their resources.

Neoclouds, which offer GPUs-as-a-service, are providing organizations with dense GPU clusters similar to those found in data centers, giving them an added advantage in processing. The technology has grown in importance since 2024, when high demand for the processors helped turn Nvidia, which specializes in designing them, into a tech-industry giant.

“If somebody says AI and cloud, I would venture to guess that a huge fraction of people are just immediately thinking of dense GPU clusters,” Blumofe said. “In the market, you’ve got these incredible evaluations attributed to the so-called neoclouds, which really focus on that kind of infrastructure.”