Small language model

A small language model (SLM) is a language model with far fewer parameters than the largest frontier models, small enough to run cheaply and quickly, and sometimes on a laptop or a phone. There’s no agreed cutoff for “small”. People usually mean models of a few billion parameters, well below the biggest models available.

Why it matters

Most AI work doesn’t need the biggest model. Classifying support tickets, pulling fields out of invoices, routing requests and summarizing short documents are narrow jobs a small model can do well, especially after fine-tuning on examples of the task. For those jobs a small model answers faster and costs a fraction as much per token.

Small models can also run where big ones can’t: on a phone, on a laptop with no internet connection, or on a company’s own servers, so the data never leaves the building.

Where they fall short

Small models know less about the world and lose the thread of long, multi-step reasoning sooner. They’re more likely to make things up when asked about something outside what they were trained or tuned for. They’re a poor fit for open-ended agent work, where the model has to plan, recover from its own mistakes and choose its next step over many turns.

Using them well

Pair them with bigger models. A common setup sends each request to the cheapest model that can handle it, so a small model does the high-volume routine steps and a larger one handles the hard cases or checks the small model’s output.

Test on your own data before switching. Benchmark scores say little about how a model will do on your particular task, and small models vary more from task to task than large ones.

How elvex relates

elvex Enterprise routes work to lower-cost models where they do the job, so routine requests don’t run on the most expensive model by default.

Bring your agent. Bring your team.