Inference

Inference is what happens when a trained AI model is put to work: it takes your input and produces an output. Every chat reply, every summary and every step an agent takes is an inference call. Training is how a model learns. Inference is the model being used.

Why it matters

For most companies, inference is the AI bill. Training a frontier model is enormously expensive, but that cost sits with the model provider. What you pay for is inference, usually priced per token: a rate for the tokens you send in and a higher rate for the tokens the model writes back.

Agents changed the math. A chatbot makes one inference call per question. An agent working through a task can make dozens or hundreds, and each call often resends the growing history of the work so far. That’s why teams that adopt agents see their token use climb fast, even when the price per token is falling.

What drives the cost

  • Model choice. A large frontier model can cost many times more per token than a smaller or open-weight model that handles the same routine task just as well.
  • Input size. Long prompts, big documents and long agent histories are paid for on every call. Prompt caching cuts the cost of resending the same material.
  • Output length and reasoning. Models that “think” before answering produce extra tokens you pay for, even if you never see them.

Speed is the other side of inference. Time to the first token and tokens per second decide whether a tool feels instant or sluggish, and they depend on the model’s size and the hardware serving it.

How elvex relates

elvex Enterprise routes work to lower-cost models where they do the job and gives admins budgets and model controls per team. In elvex itself, inference stays on each person’s own subscription: your agent runs on whatever plan you already pay for.

Bring your agent. Bring your team.