AI observability needs humans in the loop in order to excel

Are you keeping an AI on your agent?

That’s the kind of observability question required at a time when the tech industry is embracing the capabilities of AI tools and giving agentic AI more autonomy as solutions grow in complexity. For Michael Bevilacqua, VP of AI product for ETL service provider Adeptia, the question is about data oversight—a focus on telemetry that relies on the data observation being run “headless,” or without human decision-making.

“For CTOs in today’s world, there’s headless observability into the system, citations for every AI decision,” Bevilacqua said. “Those are all paths that are now required in an enterprise product.”

An approach that prioritizes that kind of headless decision-making allows for smart testing, as well, Bevilacqua added, providing full telemetric observability into a company’s ecosystem.

“What they want is a natural language interface into all the different systems, and that’s what I would be demanding from any vendor that brought products into my door,” Bevilacqua said.

Human touch. Not all companies are pushing for a completely headless version of observability and autonomy. In September 2025, for example, Kamal Hathi, SVP and GM of the Splunk Business Unit, told IT Brew that Splunk wanted a level of human control.

“You need to provide a high degree of autonomy—a high degree of agentic is another way of saying it—autonomous solutions that can go on your behalf and do things fast, rapidly respond and all that,” Hathi said. “But human control is super important; you need to have oversight of what’s going on.”

Taking notes. Review systems that rely on humans in the loop have an advantage, Indeed CIO and CSO Anthony Moisant told IT Brew at the recent Ai4 conference in Las Vegas. Having the stack automated for that type of visibility can be helpful, Moisant said, but without human oversight, you can see “massive failures.”

“I think we are many many moons away from the place where AI is doing, at least in substantive ways, the software development and the review with and then the deployment without humans,” Moisant said. “I really don’t believe that we’re going to enter a world where accountability gets shifted to AI.”