A small language model (SLM) is a language model with far fewer parameters than a large language model. There is no fixed threshold, but SLMs generally range from hundreds of millions to a few billion parameters and can run on a single GPU or on devices such as laptops and smartphones. Representative examples include Microsoft's Phi, Google's Gemma, and the smaller Qwen models from Alibaba.
Although their general capabilities fall short of frontier LLMs, SLMs offer advantages in latency, cost, and privacy and are used in on-device AI and on-premises environments. They are often fine-tuned into specialized models for narrowly defined tasks such as classification, routing, and extraction, and serve as components that reduce calls to larger models in agent systems.