Why developer review requires a hybrid strategy

There’s something to be said for a human touch, and the tech industry’s review processes are no exception.

But the need for people in the loop is running up against the reality that, with the integration of more AI tools into the tech stack, what was once done by humans is increasingly being turned over to software.

“AI is part of software development now and will continue to be—and with that comes pressure from the market, pressure from management, pressure from investors to ship faster and faster and faster,” said Darren Meyer, a security researcher with Checkmarx. “That, to me, is why we’re seeing the reduction in human review.”

Background checks. The debate over human versus AI in the review process is a critical one for the tech industry, where speed of service and innovation are often more immediately important than validation. Yet while AI review has its critics, human review simply can’t keep up with the rate at which the developer stack is moving.

“These AI agents love to do thousands of lines of code of changes,” Saoud Khalifah, CEO and CTO of AI verification startup Ciphero, told IT Brew in April. “You can’t review that as a human. It’s going to take you weeks. So, you just negated the productivity gain.”

New research from researchers at Carnegie Mellon University and Stanford, published July 1, shows the escalating importance of automated reviewing tools. Using one mid-size enterprise as a template, they found that as AI pull requests increased, the inclusion of at least one human review in the loop decreased from 89% to 68%. Meanwhile, automated AI reviews jumped from around 19% to around 84%.

Developing changes. It’s a new world for enterprises that are utilizing these tools to check work. Richard Bird, CSO at Singulr AI, told IT Brew that IT teams looking to deploy AI as part of a hybridized review solution could face a number of challenges.

“The hybrid of human review and AI review is going to prove to be the right way to build things in the future,” Bird said. “Right now, we’ve over-rotated on, ‘Let’s do all stuff with AI.’ AI is a technology; it’s a tool.”

Importantly, he added, review is a use of AI that helps power a broader technological evolution.

“I’m a full-on AI believer for what it actually is—a new technology that could actually amplify the productivity of human beings in a way that technology has been promised could do for 50 years,” Bird said.

Meyer also endorses a hybrid strategy. It’s the best possible option available, he told IT Brew, and a natural next step for enterprises deploying AI in the code review process. Humans shouldn’t be left out of the loop as the technology augments developer procedures; in a rapidly changing tech ecosystem, it’s important to leverage a full suite of tools and not to lean too heavily on one solution at the expense of others.

“Your accuracy and your reliability of these quality and security tools go up dramatically when you hybridize,” Meyer said. “That’s a pretty strong signal that hybrids of traditional tools and AI and qualified people supervising these things is probably where we’re pointed, barring some unexpected breakthrough in the field.”