How the data engineer’s role is changing

When data engineer Max Kramer was first hired by deepfake detection company Reality Defender in March 2024, he was tasked with collecting voicemail messages.

Sometimes voicemails contain computer-generated audio, but aren’t necessarily signs of a nefarious digital imposter. For example, maybe it’s your doctor’s office reminding you of an upcoming appointment.

To improve the robustness of Reality Defender’s defenses and to prevent the accidental flagging of voicemails as deepfakes, Kramer had to build a data set of legitimate voicemail messages featuring synthesized text.

Those were the days: Just good old-fashioned crowdsourcing. Not an AI in sight.

The company hired people to record voicemails and looked for available samples from telecom providers. The data set (along with a data pipeline and a single script funneling that data to the appropriate parties) was built for the specific use case, and everyone moved on.

Now Kramer spends a lot of his time thinking about the design of full systems, along with the components that plug into those systems. To create better defenses for its clients, Reality Defender needs ways to generate millions of deepfakes efficiently, cleanly, and preferably, at low cost, Kramer said.

“It’s not so much just coding whatever we need to get the next project done. It’s building scalable infrastructure that can carry with us from project to project. If we have to do something once, it’s very likely that we’re going to have to do it a lot of times, sometimes even millions of times,” Kramer told IT Brew.

Kramer helped to build one system called “Sound Forge”: a way for engineers to take a single file and apply 25 different augmentations that demonstrate common variations of recorded voices.

“We went from a single script and maybe even potentially manual intervention, which is always costly, to a cloud-native system that leverages not only our engineers’ expertise in designing optimized data pipelines, but particularly keeping in mind scale as the first order of business, given that our data sets only get bigger and bigger and bigger,” Kramer said.

Kramer is one of many data engineers who have seen their responsibilities and prominence evolve with the arrival of AI.

Meet a data pro. Here’s how Gartner VP and analyst Jorg Heizenberg defines three major data roles:

• A data engineer builds the “pipelines” that send data from sources (a CRM or ERP platform, for example) into repositories for analysis.
• A data scientist uses that structured info to crunch the numbers and gain insights to solve business problems.
• A data analyst often synthesizes the data into visual dashboards and reports, and explains those findings to business users.

Lately Heizenberg has seen data engineers moving from more of a backend developer role, responsible for extract, transform, load (ETL) functions, to an “elevated” one responsible for getting data “AI-ready.”

“Now they have become much more a part of prototyping, experimentation, and supporting AI use cases,” Heizenberg said. “The role is therefore moving more towards the front end, I would say, also placing them much more in collaboration with others.”

The workload workload. In October 2025, an MIT Technology Review Insights report, produced in partnership with data and AI platform Snowflake, made the case that data engineers “are stepping out from behind the scenes to help shape AI strategy and influence business decisions.”

The study (which surveyed 400 global business leaders, including CIOs, CTOs, and execs) revealed that more than three in four respondents agree that data engineers’ workloads are growing.

In the past, data engineers were often less visible to business partners, and thus relegated to backend functions related to extracting and transforming workloads, according to Snowflake VP of product, data engineering Chris Child. Meanwhile, data analysts and scientists (those providing data visualization and modeling outputs to business stakeholders) had more visibility with leaders.

MIT’s study, however, revealed that 72% of respondents now see data engineers as integral to the business.

Child said data engineers have to encode semantic models, or context, into data structures so AI agents can understand business questions.

“AI models demand massive amounts of high-quality, real-time data and access to a growing volume of unstructured data. This has shifted their focus from maintenance to strategy to help drive business transformation and AI development. Data is now a business’s most valuable asset so the teams responsible for maintaining and deriving value from that data are in the spotlight now more than ever,” Child wrote in a follow-up email.

New skills needed. Kramer said his role requires an added level of versatility these days: While it still involves writing scripts and spinning up virtual machines for data storage, there’s now a focus on developing secure code, communicating project issues to technical and non-technical users, and more.

Some days, Kramer might be writing database migrations. On others, he might be looking at the output from a model to see if he should tweak parameters. “I’ve found that all of us have taken on the role of engineer at some point. All of us have taken on the role of [data] scientists. All of us have taken on the role of [data] analyst,” he said.

“The more holistic your perspective, I actually think the easier it is to address these systems- level problems,” he added.