How Playbill is setting the stage for GenAI research assistance

AI is “almost antithetical” to the human, anything-can-happen experience of a live Broadway performance, according to Playbill CTO Jon Goldman, who has been with the more than a century-old company for 14 years.

While AI wouldn’t help onstage (it’s hard to imagine an announcer saying, “And the Tony Award goes to…ChatGPT!”), it might aid some of the companies that contribute to the overall Broadway experience. To that end, Goldman is trying out chatbot capabilities to support Playbill’s editorial staff, who write articles for the well-known booklets handed out before Broadway plays, as well as the 20 or so daily articles that appear on Playbill.com.

“It’s a steep curve to bring AI into this space,” Goldman said.

The CTO spoke with IT Brew about his experience testing a chatbot trained on Playbill’s specific dataset of cast and show histories. While the proof of concept hasn’t reached production yet, he sees the research assistant as potentially valuable for writers of the daily and monthly stories.

Questions, anyone?

What was the highest grossing week for any Broadway show?
When was the last time Playbill wrote about “family-friendly” shows?
Name everybody who ever played Elphaba in Wicked.

Goldman wants a “Playbill Agent Chat” to answer these questions and others that might arise during the writing process.

The idea came at an AWS re:Invent event a few years ago, Goldman said, and he spoke with business leaders from the CDW company Mission, an AWS partner supporting clients in AI deployments.

“Really the drivers were: We have all of this data. How can we start utilizing this massive curated data set?” Goldman told us.

Playbill stores performance details from the 1930s through today, along with Playbill digital articles going back to 1998. The company’s internal AI proof of concept, which wrapped just before Thanksgiving, tested how specific agents could retrieve requested data (stored in an SQL database) and send it to a Claude-based orchestrator to synthesize a response.

“My edit team is fantastic,” he said. “But they are also overworked. Anything I can do to shorten their hours and give them a better work–life balance, the longer we retain them.”

The team received “significant” funding assistance from AWS for the AI proof of concept, according to Goldman, and the tool was used by about five editorial staff members.

Getting onboard. According to a global survey from McKinsey of nearly 2,000 online respondents, less than a third of organizations deploying AI consider themselves in the “pilot” phase, another 30% are “experimenting,” while just 7% are “fully scaled” in production.

Goldman’s editorial staff raised concerns about job security and accuracy. To address fact-checking, Goldman said he communicated with Mission to make sure citations existed for all presented information. Responding to job-loss concerns, the CTO said he emphasized GenAI as a tool for the writer: “It’s not about putting people out of work, it’s not about trying to cut expenses. It’s about making your work-life easier, so that you can be more productive.”

What’s next: Determining ROI. There was one other problem for Playbill’s editorial users: The data is buried within millions of rows of an SQL database, meaning it can take time for the AI tool to find an answer. The proof of concept was meant to prove data accuracy and breadth, Goldman said.

The team also has to consider costs like data-transformation efforts—that is, turning the SQL data into a format capable of vector-based search that’s friendlier and faster for AI usage. “Now we start making the ROI calculations and saying, ‘Is it worth it?’’ Goldman told us.

In a follow-up email to IT Brew, Goldman added that he’ll need to interview writers after the trial period to see how they are using the feature. Basic “thumbs-up/thumbs-down” feedback, he wrote, also helped the dev team to see which prompts are effective.

The next step after the proof-of-concept’s first rehearsal: If costs are reasonable, Goldman said, there could be a limited deployment, which, if successful, would be followed by an organization-wide one. From there, AI presents other opportunities. Goldman is already imagining one use: If AI can read PDF versions of a Playbill, then a sales team could know which advertisers bought full-page ads, and for which shows.

“I need people to understand the tools and understand its limitations as well as its benefits, so that they are that person who can fill that role in a year, in two years, in three years where the acceleration, the speed, the curve of AI is skyrocketing,” Goldman said.