Developers fear performance evaluations based on AI data: survey

Want to scare a software developer? Skip the ghost stories and go straight to judging them on how they leverage AI at their jobs.

According to a recent report from software company Harness, 54% of developers fear performance evaluations linked to their use of AI. The report is based on a commissioned survey of 700 practitioners and managers from the US, UK, France, Germany, and India.

Stage fright? More like AI fright. Developers’ fear partially stems from the idea that leaders are watching their every move closely, according to Harness SVP Trevor Stuart.

“You only could see what I sent in as a [pull request] previously. Now, you can see every keystroke,” Stuart said. “You can see how I’m writing that [pull request] and I think that’s a little bit of the fear that comes into this from a developer standpoint.”

Nis Frome, CRO at Coderbyte, added that using AI metrics as part of evaluations may have some developers breaking into a cold sweat because AI use might not effectively reflect productivity.

“You can definitely consume lots of tokens without being a very productive engineer,” Frome said, adding that developers want to be measured on their actual performance as opposed to “some loose proxy.”

Alleviating fears. When asked what would make them feel more comfortable about including AI usage in their evaluations, 55% of developers said they want reviews to separate improvement data from performance data.

“It’s making sure that we’re using it as an optimization [and] less about a performance tool,” Stuart said. Roughly half (49%) of surveyed developers also said they want to be involved in defining AI metrics used in evaluations and transparency around what’s being measured.

“What are you collecting? Why are you collecting it? How is it being used?” Stuart said.

Trust me? Outside of agreeing to more transparency and greater involvement from developers, Frome suggests IT leaders have faith that their developers—who he said were likely hired because they were deemed qualified for their jobs—are using AI resources as needed.

“Are there developers out there going to purposefully move slower and not use AI? [That] doesn’t really make sense to me. They’ll probably use it when it makes sense and not [use] it when it doesn’t make sense,” Frome said. “So, if you’ve given them the resources and you’ve given them the training, hopefully they’ll do their job.”

He added IT managers may have bigger issues to worry about than whether a developer is relying on AI for their work.

“What’s really concerning right now is we’re talking a lot about AI productivity, but what we’re not talking about is some of the biggest platforms in the market right now, like GitHub, are having countless outages,” Frome said. “I think we might need to shift our focus to performance around that and not around proxies that may or may not actually matter.”