Should enterprises invest in SLMs? Experts are mixed

What if we took a Large Language Model (LLM) and made it…smaller? Experts are mixed on if such Small Language Models (SLMs) could deliver LLM-sized results at a reduced cost. For IT pros, the decision to use SLMs could have a sizable impact on their organizations.

Tom Bachant, the CEO of Unthread, a company that provides AI-powered technical solutions and assistance via Slack, doesn’t see SLMs as the answer to organizations’ needs for chatbots and other AI tools, especially with consumer expectations so high, thanks to LLMs like OpenAI’s ChatGPT and Google’s Gemini.

“You need to give a large language model access to some of this data, and that’s not inherently a bad thing,” Bachant said. “Companies [are] already doing this today. They connect all their tools through Workato or Zapier, or these different tools where there is access and that’s shared across different platforms.”

If you didn’t know…Small language models aren’t just fun-sized versions of LLMs. As described by Hugging Face, they’re designed to operate on resource-constrained environments like embedded systems, low-power computers, and smartphones.

SLMs are “shrunk” through techniques like knowledge distillation, which is when a smaller model is trained on knowledge transferred from a larger model, and quantization, which is when one reduces numerical values used in calculations, like only using integers. However, these models are more limited in their knowledge and capabilities than LLMs, and potentially less accurate. 

So, why would we use them? SLMs can mostly support chatbot, edge computing, language translating, and content generation applications.

Dorit Zilbershot, the global VP of product management and AI at ServiceNow, told IT Brew that SLMs can help with specialized actions that require accuracy, such as a chatbot that uses specific company terminology. She emphasized, however, that the need for this is case-dependent. 

“What I always tell my customers, that [SLM] knows how to do [a specific task], but it will never be able to write you a poem in French,” Zilbershot said. “An LLM will be able to give you some summary of an incident, maybe not exactly tailored to how [IT service management] want to see it, or not exactly fully understanding IT terminology or what an incident is even…but it will also be able to write you a poem in French.”

For IT professionals, that specialization might be enough to justify the implementation of SLMs into their tech stack.