How a prompt engineer fields tax questions and gets AI answers

In March, when Greg Brockman, president and co-founder of OpenAI, wanted to demo the exciting power of GPT-4, he turned to one of life’s least-exciting inventions: taxes. It caught Ken Priyadarshi’s eye that a language-learning model could seemingly take heaps of policy and output the answers to burning payment questions, like:

What is a couple’s tax deduction in 2018, a year with special rules? Or,What is a couple’s total liability?

The OpenAI team showed how GPT-4 could answer both questions swiftly.

“They were thinking the same way that I was thinking in the context of how this could be applied,” said Priyadarshi.

Priyadarshi is a prompt engineer—specifically, a tax prompt manager at the professional-services firm EY, someone whose job is to translate client questions into worded queries that a language-learning model understands and answers. The role—a position requiring command of both language and data—is an emerging and valuable one, especially if a prompter has specialized knowledge.

“I don’t think of myself as a prompt engineer, I think of myself as a tax prompt engineer…I understand the domain-specific systems and data that our firm is working with. So, my goal is to design prompts or prompt families or prompt databases that are specific to that data set; that is valuable,” said Priyadarshi.

Be prompt! While some see hype in the career of prompt engineering (and some industry pros believe prompt engineers may be replaced by the very AI they’re training), LinkedIn recently told Time that posts referring to “generative AI” increased 36-fold in comparison to last year—a stat suggesting that companies increasingly want professionals who work with language-learning models.

When asked if prompt engineering would emerge as a profession, Nick White, data strategy director at the consultancy Kit + Carta, put the chances high: “1000 percent.”

“There’s so much fine-tuning to be done to make some really valuable, bespoke models and applications of this technology that you need those people focused on that,” White said.

Model behavior. EY, in June, announced its own Tax Copilot educational tool. The prompts designed for EY’s AI systems are often client-scenario-based prompts, run on private models on top of Microsoft’s Azure OpenAI infrastructure. A prompt may be similar to a CFO’s queries about tax or risk data, and then Priyadarshi has to fine-tune the question to fine-tune the model and its answer.

“For example, what prompts might a tax lawyer want to use to generate a simple legal template?” Priyadarshi wrote in an email. “Or what prompts might a call center worker use to query jurisdiction-specific data about payroll policy? What questions might a CFO ask to get an executive summary on a recent cash flow analysis report? What questions might a COO ask to understand their supply chain risk in a country?”

Human-in-the-loop reviewers are tasked with catching errors proactively; typical fixes for incorrect answers involve retraining the model with more nuanced examples. And the answer may take a few tries, especially if the data needs to be presented in a certain way.

“The model might be dumbfounded, right? Then, you have to be a little more specific,” said Priyadarshi, who may need to “coach” the model to pull columns, or present the data in tabs, or output a JSON data structure for engineers.

Priyadarshi was a strategy analyst who then moved into tax data engineering—a job that often requires coding to address company questions. A prompt engineer can do the same problem-solving, minus the code.

While no-code prompting has a way of democratizing data analysis, a generic prompter may not know a given field’s nuances, like how Priyadashi often sees errors when language models handle value-added tax (VAT) invoices. A specialized prompter can get ahead of VAT.

Before Brockman demo’d GPT-4 in March, he said: “GPT is not a certified tax professional, nor am I, so you should always check with your tax advisor.”