In its recent announcement of a ChatGPT agent that “thinks and acts,” OpenAI recommended several ways a user can deploy the technology:
- Look at my calendar and brief me on upcoming client meetings based on recent news.
- Analyze three competitors and create a slide deck.
IT Brew spoke with Bret Kinsella, GM of Telus Digital’s Fuel iX generative AI platform, about what can go wrong in those situations and what steps IT pros can take to address agents gone awry.
Gone corporate. OpenAI announced in September 2024 that the corporate userbase of its Enterprise, Team, and Edu products had reached 1 million.
OpenAI’s July 17 announcement of “agent mode” brings together two capabilities with the familiar ChatGPT tool:
- Operator: a feature supporting autonomous browser interactions
- Deep research: an agentic tool pulling analysis
Put it on the calendar. When an agent creates a briefing by accessing a calendar, one threat that concerns Kinsella is prompt injection—a command hidden in, say, a calendar entry that enacts a malicious activity like data retrieval. Another consideration, Kinsella shared: A ChatGPT agent may someday connect with other third-party agents, which then may access sensitive information on those calendar invites—a doctor’s appointment or other personal details—or other applications. “I don't think I’ve met a CISO yet who has a clear plan on how to address something like this,” Kinsella said.
OpenAI says connectors are available for Enterprise users; options include GitHub, Gmail, and Microsoft SharePoint. Customer connectors are also possible, but “are not verified by OpenAI and are intended for developer use only,” OpenAI notes.
To address calendar conundrums, Kinsella recommends starting with policy or configuration—like prohibiting the agent from sending calendar-related emails, or structuring the agent to access calendar data in read-only mode, in this case.
Hit the deck! When an agent creates a slide deck, the data can be flat-out wrong. IT pros should create a limited dataset, Kinsella said—say, only using information from a consultancy’s subscriptions, or blacklisting URLs (like ones that don’t use “Https”). The main defense may just be a classic: checking your work.
Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
“We should all just have some personal responsibility about using these things,” he said.
When asked how a company can prevent agentic prompts from causing unexpected data exposures, OpenAI declined an interview, but pointed us to its Help Center page, where it lists measures like:
- using settings to disable connectors unrelated to a given task
- employing “takeover mode”—a private option that deploys ChatGPT’s browser when handling sensitive information
- avoiding open-ended requests like “take a look at my email and feel free just to take every action to move the discussions forwards…”
- using data-control settings to clear saved passwords and keep sessions authorized
One tech pro proceeding with caution is the OpenAI CEO himself, Sam Altman, who wrote recently on X that the “experimental” tool should not be tried “for high-stakes uses or with a lot of personal information until we have a chance to study and improve it in the wild.”
Tom Gould, senior architect at consultancy West Monroe, recommends enterprise agent users limit functionality, restrict permissions, and enable read-only access to protect data. Also: test in non-production environments.
How do you test a technology, however, that supports open-ended inputs and outputs? Today’s security tools, according to Kinsella, aren’t designed to interrogate open text boxes in real time and at a massive scale. In other words, today’s researchers have their work cut out for them.
“There are not enough skilled red teamers to interface with all of these systems out there,” he said.