Help desk pros notice employees going to LLMs first

Chatbots are becoming first stops for employees with “how to”-style IT questions.

Although such queries potentially free overworked IT pros from having to ask, “Did you try restarting?” for the hundredth time this week, this reliance on LLMs may also blind the help desk to problems brewing within the organization’s tech stack.

Robert Rohrman, SVP of IT infrastructure at IT certification and training body CompTIA, has seen an increase in employees’ turning to LLMs for their questions. While a help desk go-around may save tech-support time away from the queue, he notes, there are downsides.

“You want to make sure that people using chatbots are getting proper information,” Rohrman said. “There are ways for corporations to control that.”

Bot have we here. AI answer engine Perplexity examined 85.5 million recent queries in its enterprise model and found:

• Almost 1 in 5 (18%) Perplexity search-engine queries were technology questions.
• More than three-quarters (77%) of those tech questions “resemble work an IT team would normally handle as tickets.”
• Those questions clustered around software usage (for example, “Why can’t I see this file in SharePoint?”), troubleshooting, networking, security, and cloud queries (like permissions, regions, and performance issues in platforms like AWS, Azure, Databricks, or Snowflake).
• The same questions come up again and again. In a given month, under a third (30%) of technology topics involve two or more people within an organization asking about the same issue.

The risks. For IT leaders, that “invisible stream” of LLM-directed queries, according to Perplexity, carries both risk and opportunity. “If these questions remain unseen, systemic problems stay buried and tools that look successful in dashboards may be quietly undermining productivity,” the company’s study authors wrote. ​

Aside from systemic failures, sidestepping the help desk could mean less visibility into potential leaks of sensitive data, and may give the IT team a false impression that no news about IT issues is good news about IT issues.

What to do. Amit Patel, SVP of consulting services at tech-services firm Consulting Solutions, told IT Brew via email that he has seen an increase in employees defaulting to AI tools before submitting a help desk ticket, especially for common issues like troubleshooting software errors, configuring tools, or navigating internal systems.

Patel recommends:

• IT teams deploy sanctioned AI assistants that are integrated with internal knowledge bases and ticketing systems.
• Capture the data. AI interactions should feed back into IT service management (ITSM platforms) so teams can identify patterns, track recurring issues, and continuously improve knowledge articles and root-cause resolution, he advised in an email to IT Brew. (Perplexity, in its white paper, offered similar advice: “Treat AI query logs as IT telemetry” to provide signals of where tools, documentation, and onboarding are falling short.)

“The answer isn’t to fight the trend—it’s to embrace and operationalize it,” Patel wrote.

Rohrman advised help desk pros to build onto their existing enterprise-grade LLM, and make that option the go-to instead of an external, unsecured model. He recommends using that sanctioned LLM’s available connectors to pull from curated document repositories, like collab-tools SharePoint and Confluence. Also, track which queries are successfully solved with automation—and improve documentation or refine configurations for less successful interactions.

Patel sees the reliance on LLMs as a chance for IT pros to focus on more complex problems like automation, governance, and experience design.

Noel Temena, help desk pro and founder of B2B consulting and training company Deductive Lab, similarly envisions the trend forcing help-desk pros to expand their knowledge beyond the basic troubleshooting of Tier 1 support and to be more holistically aware of IT environments.

“There are going to be less issues with, ‘How to do this? How to do that?’” he said. “It’s going to be more technical.”