Want an agent? IBM guides clients through AI guardrails

Consider this: You’re looking for a water bottle that exudes your style. Gatorade currently allows you to custom design your water bottle with a prompt of “anything you can imagine.” (“A panda flipping pancakes,” the site suggests.)

What could go wrong—besides that proverbial first GenAI pancake?

At a June 12 meeting in IBM’s Manhattan headquarters, the company’s agentic experts considered the disastrous outcomes in a custom-bottle scenario—with no technical safeguards in place.

For one, a large language model (LLM) has to be ready to field offensive prompts and offensive outcomes in such a public application.

You want to filter and test the outputs and inputs to ensure they are customer-appropriate and in line with brand guidelines, Manish Goyal, VP and senior partner, global AI, and analytics leader at IBM Consulting, said. A technology must reject displeasing prompts before they even hit the LLM.

“You want to have guardrails on what I will allow or not allow,” Goyal said.

Before an agentic or generative AI tool goes live, IBM’s team offers workshops based on client needs to help orgs determine priorities and mechanisms to guard against potential negative impacts. (PepsiCo, Gatorade’s parent company, went through such an exercise, Goyal told us. Lily O’Brien, global comms lead for IBM, later shared in an email that IBM worked with Pepsi to establish a “Responsible AI framework” and set of governance policies.)

Goyal and Phaedra Boinodiris, IBM Consulting’s global leader for trustworthy AI, walked IT Brew through the data-governance drills in a theoretical scenario: a hotel wants an AI agent in guests’ rooms to act as a concierge and provide activity suggestions.

Personas. First, a consultant guides the client through all the people impacted: a front-desk employee, a child guest, an AI engineer, a restaurant owner, a litigator, to name a few.

They’re then placed into categories of:

• Data subjects (a restaurant owner who has their data in the AI system)
• Decision subjects (a guest whom the decision is about)
• Pedestrians (those indirectly impacted, without a direct role in shaping system decisions)
• Decision-makers (people using the AI to support a decision)
• Data makers (people who input the information as data into the system)
• System builders and influencers (people actively involved in developing and setting direction for systems)

“It’s so critically important to have the groups of people doing these exercises be truly reflective of the diverse communities that you’re serving, but also be multidisciplinary,” Boinodiris said.

Let’s play risk. Once that’s mapped out, a team dedicated to use cases, like procurement or a business unit, considers “secondary positive effects” (new partnership opportunities with a scuba-diving company) and “tertiary negative effects” (liability issues related to data sharing, or wrong and frustrating outputs).

Following a categorization of risk, the design team reviews agreed-upon company priorities—say, data privacy—and vote on “baseline,” “enhanced,” or “vigilant” guardrails.

A team may decide that “explainability” and “observability” are important aspects for the hotel AI deployment. That could mean basic security logs for observability (baseline) or live dashboards with anomaly detection (vigilant); each has varying costs. Explainability could mean simple explanations for what data was used to train the model (baseline), or it could mean links to evidence as part of its output (vigilant). “Enhanced” practices are the middle option.

“This gives people the language to use, both with the builders and with the buyers, so that they can understand when you use the word ‘fairness,’ or you use the word ‘explainability,’ what it actually means in terms of what needs to be built,” she said.

And every company can determine its important criteria: explainability, observability, or otherwise.

“If I have an AI that’s predicting the movements of ants on an ant farm, data privacy is not applicable,” Boinodiris said.

Then, the “grand finale,” according to Boinodiris, involves presenting an org’s AI council the business use case, the unintended effects or risks, and the approach for mitigating them.

A company’s requirements may change as the unexpected occurs. Harvard Business Review recently shared how orgs are not prepared for “ethical nightmares” related to disasters like IP violations, discriminatory outputs, and privacy exposures. What could go wrong? In Gatorade terms: Anything you can imagine.

“It is always the right organizational culture that is required to curate AI in a responsible way, one that you know approaches this with humility and a recognition that this has to have constant reflection,” Boinodiris said.