Energy per token
Each answer a model gives costs a little electricity, and how much depends on what you count: the chips, the rest of the server, idle machines and the building's cooling.
Read first: Power & cooling for AI data centres.
D1 · StoryFor young children and total beginners
Every word the model writes uses a tiny bit of electricity.
Most goes to the chips doing the maths. Some goes to the rest of the computer, to machines waiting for work, and to cooling the building.
A bigger model, or a longer answer, uses more.
Like: A taxi ride's fuel: the engine, but also the air conditioning and the drive back empty.
D2 · PictureFor curious beginners
Energy per token is the electricity used to produce one token. Measured only at the accelerator it looks smaller; a fuller count adds the host CPU and memory, machines kept idle for spare capacity, and building overhead such as cooling and power conversion.
That building overhead is captured by PUE (power usage effectiveness): total facility energy divided by the energy used by the IT equipment. A PUE close to 1 means little is lost to cooling and conversion.
Energy per token falls when the same chips serve more tokens: bigger batches, smaller or quantised models and newer, more efficient chips all help. Longer answers, reasoning steps and larger models push it up, so totals can still grow.
Like: A taxi ride's fuel: the engine, but also the air conditioning and the drive back empty.
Where it breaks: A taxi carries one fare at a time. A server can batch many users' requests together on the same chips, so the energy per token depends heavily on how busy the hardware is.
D5 · FrontierFor experts
There is no agreed standard boundary or method, so published figures are hard to compare. Real numbers depend on batch size, prompt and answer length, hardware and how busy the servers are.
Lower energy per token does not guarantee lower total energy: cheaper tokens can mean far more tokens, especially for agents and reasoning models that write long hidden chains of steps.
Sources
- Measuring the environmental impact of delivering AI at Google Scale (Elsworth et al.) (2025-08-21)
- Power Hungry Processing: Watts Driving the Cost of AI Deployment? (Luccioni et al.) (2023)
- From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference (Samsi et al.) (2023-10-04)
- Best Practices Guide for Energy-Efficient Data Center Design (US DOE FEMP / NREL), PUE definition (2024)