AI agents are making vendor switching messier

When ChatGPT first spurred a run on enterprise AI licenses, Emmanuel Frenehard resisted the peer pressure.

Frenehard, the chief digital officer at pharmaceutical company Sanofi, said he didn’t want to get locked into contracts that would ingest the company’s lifeblood data. And Microsoft Copilot didn’t seem like much of a productivity boon for the cost at the time.

Instead, Frenehard worked with Snowflake and a Snowflake-backed startup called Elementum to build a central data lake and an internal AI system called Concierge. In the next two years, Frenehard wants 80% of the company’s IT tickets to go through Concierge.

But even given Frenehard’s efforts to avoid these contracts, moving away from outside vendors involves some tricky navigation, especially as each platform increasingly offers its own stable of agents.

“We’re working on new ways of how you prepare the disentanglement of it,” Frenehard said. “Because the last thing you want to do is to be locked in for a long term.”

As companies spin elaborate webs of agents and other AI workflows, the logistics of switching agent vendors or underlying foundation models has become more complicated.

‘A lot messier’

Whereas traditional software is deterministic—a given input always yields a set outcome—a foundation model is a black box that requires extensive evaluation to ensure that the AI won’t do something unexpected. To further complicate the already cumbersome process of vendor switching, best practices around agents are still in flux, token costs are uncertain, and it’s difficult to replicate outcomes in fine-tuned models.

“It’s a lot messier than it used to be traditionally,” Gartner VP Analyst Kjell Carlsson said. “There’s an element of uncertainty that we don’t know how to do migrations in the same kind of way…We’re just not even sure what we need to migrate.”

Agents are defined by overarching prompts that instruct them on how to interact with tools and their surrounding environments. But it’s hard to know how different models—even those from the same provider—will interpret the same prompt, according to Forrester VP and principal analyst Mike Gualtieri.

“You cannot assume that it’s going to react to the prompt in the same way,” Gualtieri said. “Could it do something drastic? Like, if it was a pricing agent, could it lower a price, and all of a sudden you sell 10,000 of an item at a price you didn’t expect?”

At revenue platform 6sense, chief product and technology officer Kimberly Bloomston said the company evaluates each new agent it rolls out across seven different areas, with tests that include pass-or-fail contracts and full trace coverage.

“That eval bank is also what turns swapping a model from a leap of faith into a regression test,” Bloomston said in an email.

The elements that are hardest to switch—“fine-tuned models, embeddings, and vector indexes, prompt scaffolding tuned to one model’s quirks, and your evaluation history”—are kept to a minimum.

Agents everywhere

As companies build out thousands or even tens of thousands of agents for all kinds of tasks, IT is having to figure out the rules for documenting and organizing them and potentially switching out vendors on the fly, according to Carlsson.

“A lot of IT folks are just overwhelmed because even that reference architecture for an AI agent isn’t super well established,” Carlsson said. “Just even knowing and creating taxonomies: What kind of AI agents do I have? What are the different components that they’re using?...If I were to switch out a vendor, which one should I switch out to? All of this is then blurred with the conversation around the exploding ecosystem of different tools.”

Even traditional enterprise software platforms now offer agents of their own as well, which can make subscription lock-ins more complex. Frenehard said Sanofi does use some of these agents, but often they are confined to just one software platform.

“You might be able to help in that world, but that world is kind of finite. It’s a silo,” Frenehard said.

Unlike with traditional software, which demands a high up-front cost burden but comes with stable and predictable future costs, AI agents are easy and cheap to initiate, according to Carlsson. But that ease comes with a drawback for IT pros tasked with maintaining these agents, which are harder to switch down the line.

“In the AI space, it’s now so easy to get started with any tool and any platform out there, both because they’re cloud-hosted, but also because they’re usually subsidizing the usage of them,” Carlsson said. “The problem here is that everybody around the organization can afford to get started, and they are. And then now you’re on the hook for supporting and maintaining these going forward.”

Keeping it simple

In that environment, Gualtieri said one key is to make the prompts governing agents as simple and discrete as possible, even if that means decomposing one agent into multiple functions.

“Pay very special attention to how you write the prompts in the agents,” Gualtieri said. “Anticipate what the impact could be. Be as precise as possible in those prompts [in terms of] how you want it to behave and what you want it to do. The more precision, the more likely it is that models are going to behave in a similar fashion.”

Carlsson also cautioned against entering into any long-term AI contracts, given the pace of change in the space.

“Start thinking about exit when you’re writing the contract. I don’t think a lot of these contracts really have spent much time going in and putting clauses about the degree to which they can exfiltrate their data and models and agents from these systems…Plan for it and contract for it up front.”