Are LLMs coming for cybersecurity jobs? AI/ML researchers don’t think so

Are large language models (LLMs) like OpenAI’s ChatGPT, Amazon CodeWhisperer, and Google Bard coming for cybersecurity researchers’ jobs?

The short answer? No. The longer answer? Not exactly. LLMs aren’t going to take security jobs left and right, according to some of the researchers actually using them. Instead, it’s more likely that the use of AI/machine learning (ML) tools will streamline time-consuming grunt work and let pros focus their efforts on more complicated tasks.

Fuzzy outlook. Fuzz testing is an automated vulnerability detection process that submits random, unexpected, and/or malformed inputs to test targeted code for errors like crashes and memory leaks. LLMs can help scale fuzz tests further by automatically generating and refining new fuzz targets.

At DEF CON 2023 in Las Vegas in August, vulnerability researcher Xavier Cadena presented FuzzForest, an open-source tool that automatically writes, fixes, and triages fuzz tests on Python code. FuzzForest helped Cadena quickly identify over 70 vulnerabilities across 20 of the most popular open-source Python libraries.

Cadena told IT Brew he intended FuzzForest, which combines “coverage-guided” Python fuzzer Atheris and ChatGPT, to be “entirely automated” and only require a library URL to function.

One big challenge was hallucinations—hard caps on input length can limit how well the LLM can view submitted code in context, resulting in scrambled outputs.

“I found that the best way is just to extract the singular function with enough context,” Cadena said.

Cadena says FuzzForest-like tools come in handy for security researchers, who tend not to be “the strongest at writing code.”

“LLMs won’t give you the answer, but teamed up with a domain expert…you’re able to make a lot of progress,” he told IT Brew.

Cadena said it cost roughly $50 to have FuzzForest write around 700 functions for the 20 repositories, which might otherwise “take an engineer a year just to understand what’s going on.”

AI assistance. Other applications of LLMs include malware deconstruction via software reverse engineering (SRE) frameworks like IDA and Ghidra, API analysis, and query generation for threat hunting detection tools like Yara.

“Finding the discrepancies might be hard/require expertise not much different from writing it to begin with,” Vaitzman and Furman added.

Patrick Ventuzelo, CEO and founder of Fuzzing Labs, recently used GPT-4 to help discover zero-day exploits. He told IT Brew LLMs are “really to be used in addition to existing fuzzing framework.”

“The main issue that a LLM would get to find a vulnerability is that it will not get execution feedback, or it will not be able to get the full context of the execution,” Ventuzelo said.

“Leveraging an LLM to generate more interesting inputs that will allow us to go deeper, that’s basically the best way to use it.”

Searching for cybersecurity pros. None of the experts who spoke with IT Brew thought LLMs were a threat to cybersecurity jobs, at least at the moment.

Vaitzman and Furman noted cybersecurity is a “zero-sum game,” where new tools for attackers constantly scale up the needs of defenders. For example, LLMs could automate some security operations center functions but make it easier for organizations to start their own.

“This may lead to some reduction in the current task force, but at the same time, might lower the entrance bar for new personnel,” they wrote, which is “much required due to the potential increase in load of attack due to ease of process automation.”

AI and ML tools will more likely mean cybersecurity professionals acquire hybrid skills than lose work, Vaitzman and Furman wrote. For example, they may have no choice but to adopt AI tooling to “combat newly automated attacks,” they added.

Cadena pointed to the extant shortage of cybersecurity talent, saying AI tools won’t replace them but act as a force multiplier.

“You still need the art of fuzzing,” Cadena told IT Brew. “There’s an explosion of software in general…So obviously, there will be more bugs, but they’re gonna be much harder to find.”