Is AI decimating the bug bounty industry?

These days, finding a high-quality bug bounty submission can feel like looking for a needle in an ocean of AI slop.

It’s been a rough year for public bug bounty programs. In May, SQL database Turso announced it was “retiring” its own bounty program due to a rise in AI slop, or low-quality submissions that exhaust resources.

“The main problem of course is that it costs the slopmaker perhaps a minute to generate their submission,” Turso wrote in a May 12 blog post. “But it costs us hours to read, understand, and engage with them. And they can be generated at a semi-infinite pace.”

Slop it like it’s hot. Turso isn’t the only one to shutter its bug bounty program because of AI slop. In March, HackerOne announced it would pause new submissions to its Internet Bug Bounty program, which launched back in 2013. In a statement, the cybersecurity platform said it would reevaluate the structure and incentives needed to further its goals of both “discovery and remediation” as AI-assisted research accelerates vulnerability discovery.

Similarly, Curl ended its bug bounty program, which started back in 2019, in January after founder Daniel Stenberg said he was getting “more AI slop than ever before.”

Too little, too late. Beyond being a nuisance, Searchlight Cyber Chief Research Officer Shubham Shah told IT Brew that AI slop submissions to bug bounty programs can divert attention away from legitimate submissions.

“That means that all the really talented or skilled hackers are finding that this new age of AI is actually causing…these companies to not know about these higher-risk or critical-risk issues in time,” he said.

Changes TBD. There’s a lot at risk if public bug bounty programs can’t combat the current slop problem, according to Shah.

“If those solutions to these problems are not built out thoughtfully or pragmatically, then your experience as a hacker may degrade significantly, which means that you will be less motivated to spend time on those platforms as someone that’s bringing real skills to the table,” he said.

Some programs have already started to make adjustments. On May 15, GitHub redefined what it considers a “strong submission” from researchers and said it would reward low-risk findings with swag as opposed to a monetary payment. The month prior, Google announced changes to its Android and Google Devices and Chrome Vulnerability Reward Programs, focusing more on issues with the highest risk to its users and prioritizing “categories that remain more challenging for automated AI tooling to find.”

“We now consider the most effective reports to be concise, containing only a reproducer and the necessary artifacts to help us validate and route the issue,” the company wrote in an April 30 blog post.

Daniel Andrew, head of security at cybersecurity company Intruder, told IT Brew these are “sensible” ways of dealing with the slop problem.

“They’re using that to try and combat slop reports because slop reports cannot have a good reproducer in them,” Andrew said. “It might preclude certain bugs and concerns and vulnerability classes from being reported, but those would typically be the less important ones anyway.”

Ultimately, Andrew said, programs need to find a way to “evolve their triage processes” for the AI era: “They’re gonna have to find a way of probably using AI as well to solve the problem, but doing it in a way that doesn’t compromise quality or compromises quality as little as possible.”