Model APIs & routers
Paying per token to send requests to a hosted model, either directly to a provider or through a router that picks among many providers.
Read first: Tokens & tokenization.
D1 · StoryFor young children and total beginners
Most people never run an AI themselves. Their app sends a message over the internet to a company that runs it, and pays a tiny amount for each word.
A router is like a taxi app. You ask for a ride, and it picks which company sends the car.
Like: A taxi app that picks which company sends the car.
D2 · PictureFor curious beginners
A model API is a web endpoint: you send the conversation as JSON and get the reply back, often streamed token by token. You pay per token, usually quoted per million, and input and output tokens are often priced differently.
Many providers and local servers copy the same request format, often called OpenAI-compatible. Switching can be as small as changing the base URL and the model name.
A router or aggregator sits in front of many providers. One key and one bill reach many models, and it can send each request to whichever provider is cheaper, faster or currently up.
Like: A taxi app that picks which company sends the car.
Where it breaks: Any taxi gets you to the same place. Two providers serving the "same" model can differ in quantization, context limit and supported features, so the answer you get may depend on which one the router picked.
D5 · FrontierFor experts
"OpenAI-compatible" is a convention, not a formal standard. Tool calling, caching, reasoning settings and error handling still differ between providers, and those differences break apps in subtle ways.
A router adds a hop that also sees your prompts, so data handling and logging policies matter for both the router and the provider behind it. Per-token prices also hide costs that depend on usage, such as long cached prompts or reasoning tokens you never see.
Examples as of 2026-10-13
- OpenRouter (router)
- vLLM and Ollama (local servers with OpenAI-compatible APIs)