IT pros turn to live tests to find true talent

At Abby Kelley Foster Charter School in Worcester, Massachusetts, even the candidates for IT jobs have to take a test.

Gabriel Beltran, the school’s IT director, was looking for a data manager. Following interview calls, he gave candidates two files: one with names and email addresses, another with grades and homeroom assignments. Then he presented them with a merged report that he wanted them to recreate. Beltran looked on as they went to work.

“I decided to include the test, because I don’t want to hire somebody and then find out a month or two months later that they didn’t really have the skill,” Beltran told IT Brew, adding that he let interviewees use whatever tools they prefer to create that merged document.

That kind of in-person assessment is especially important as AI offers ways for candidates to polish a résumé, ace a take-home test, or quickly gain answers via chatbot.

What’s out there. Generative AI has reshaped how quickly candidates can spiff up their CV, with tools that create résumés (often tailoring them to essential keywords), build LinkedIn profiles, and enable applying to multiple jobs at the speed of bot.

For coders, interviewing can involve coding copilots and even sneaky screen overlays—tools that allow candidates to receive AI-powered answers without a hirer knowing. Some available products offer live transcription, along with suggested ideas for what to say next.

Bring it in! A recent survey from talent solutions and business consulting firm Robert Half found that around two-thirds (67%) of US HR leaders say reviewing AI-generated applications has hampered their hiring; one in five reported delays of more than two weeks. Some organizations, like the Association of Chartered Certified Accountants, reportedly require candidates to take assessments in person to negate any AI assistance.

Check please! Christine Belmonte, president of technology staffing at staffing and advisory firm the Planet Group, and Joe Qualiato, recruitment team lead at the company, have to verify candidates’ skills and identity. They perform standard background checks, examine work histories, and make sure they have a video interview with a hiree before sending them to a client (as well as a mandatory live introduction on calls, to prevent an impersonator from attending).

For projects mentioned on a résumé, both stressed the importance of exploring the project “end to end”—what problem did the candidate diagnose, what did they change, how they verified a fix, and so on.

“If they can’t get specific, that’s when you know you have a gap in that execution,” Belmonte said.

Qualiato likes to throw a hypothetical “roadblock” at a project seen on a résumé, then watch how a candidate responds. “If someone’s been through it, they’re going to know how to calm [themselves] and then answer a question and walk you through it,” Qualiato said. “Have they actually really done it, or were they just part of the team that did it? Were they the one that handled it, controlled it, and figured it out? Or were they just assigned a part of that problem and fixed their part of it?”

Pop quiz, hot shot. Chris Daden, chief technology and AI officer at talent success and candidate assessment company Criteria Corp, used to give engineering interviewees a take-home test of sorts. They’d have four hours to code a project—a scoped build representative of real, on-the-job work. That doesn’t happen anymore, thanks to today’s large language models (LLMs).

“We just can’t do that anymore, because when they take it home, it’s just completely written by Claude Code, or ChatGPT Codex, or whatever it is, and it’s a completely useless signal,” Daden said.

He sees résumés, degrees, and GitHub stars as old signals, versus more “durable” signals like resilience and critical thinking, which Daden can evaluate via a 90-minute live coding experience with an engineer. It’s less about the syntax and more about problem solving.

Daden noted he’s fine with candidates using AI to assist their work: “I’m asking them along the way, like, ‘Great, you just wrote 40 files of code in six minutes. How are we mitigating for the risk of low-quality code that the AI generated?’”

Back to school. Beltran has always been one for presenting a candidate with a problem. (In 2025, he told us he sometimes asks systems admin hopefuls to “draw a network.”)

Professionals vying for emerging job roles spots will likely have to arrive with more than just a polished résumé; they’ll need to provide some evidence they can handle the job.

“It is important to see it in real life, to see if they can do it,” Beltran said.