At DEF CON 31, deepfakes go real time

Former CyberArk CEO and current chairman Udi Mokady spoke at DEF CON 31 in August, except he didn’t.

What at first glance looked like Mokady was actually CyberArk security research group manager Gal Zror, who demonstrated a new process that uses existing generative AI tools to impersonate anyone’s face and voice—in real time. During his talk at DEF CON, Zror also impersonated conference founder Jeff Moss live on stage, telling the audience, “I’m Jeff Moss, founder of DEF CON. I hope you enjoyed this talk because right after it, DEF CON is officially canceled.”

Zror’s technique, “DEep Fake CONversation for VIDEO and Audio in Real-Time” (DEFCON VIDEO-ART), could be replicated by anyone with patience, a few audio and video samples, and access to a powerful graphics card or server. It’s proof that the age of the real-time deepfake is here, which means phishing attacks using similar methods can’t be far behind.

Voice cloning is already good enough to trick automatic speaker verification systems used to protect systems like bank customer service lines in some cases, and the toolkit for attackers to blow past voice authentication keeps growing. Technology for video cloning remains less advanced, but has rapidly caught up, aided by the massive amount of training data available to be scraped from social media.

“I started by mapping all the different tools and what I actually discovered is that the technology is already there,” Zror told IT Brew. “There are many projects in GitHub, all open source, that provide just about everything.”

Zror, who says he is “not a machine learning expert whatsoever,” learned how to create and refine real-time deepfakes over the course of a few months. The software involved includes:

• DeepFaceLive, a realtime version of an AI deepfake tool called DeepFaceLab
• RVC Project, an open-source AI voice conversion tool
• Voice Changer, a real-time voice conversion client that supports RVC
• Screencasting software like AV Audio or OBS Studio to broadcast the deepfakes as a virtual camera stream

Zror described the process of training the AI models as challenging to learn, but easy to automate when all the working parts are in order. DEFCON VIDEO-ART is a “fancy bash script” that automates the entire process with pretrained models, best practice configuration, and all the necessary environmental dependencies. Audio takes just a few hours to set up, while the video component takes a few days using a commercial GPU.

“I just need a few minutes of video samples and one minute of audio,” Zror told IT Brew, “ which is pretty scary.”

There are still major limitations. The tool struggles with emulating hairstyles—but Zror ran the video feed through Snap Camera, which allowed him to pose as Mokady without shaving his head.

Zror also said his method has a slight lag time and can run into difficulties if objects like hands or glasses obscure the face, although an attacker could potentially explain any artifacts or glitches away as the result of a low-quality connection. He told IT Brew those obstacles are temporary and potential solutions like automated deepfake detection are “in the very early stages.”

“As far as I know, there’s no real commercial product that you can just use,” Zror said. That means deepfake defense will have to rely on security awareness and culture.

“When it comes to enterprises, it’s even more important to educate everybody to just be very, very suspicious,” Zror added. “You need to say, I’m sorry Mr. CEO, I know you’re the top manager, but you know I need another verification.”