Energy per token

Each answer a model gives costs a little electricity, and how much depends on what you count: the chips, the rest of the server, idle machines and the building's cooling.

Layer 1 · Physical & Energy · Run time · reviewed 2026-10-30

Read first: Power & cooling for AI data centres.

D1 · StoryFor young children and total beginners

Look at the stack: which part do people usually forget to count?

Every word the model writes uses a tiny bit of electricity.

Most goes to the chips doing the maths. Some goes to the rest of the computer, to machines waiting for work, and to cooling the building.

A bigger model, or a longer answer, uses more.

Like: A taxi ride's fuel: the engine, but also the air conditioning and the drive back empty.

D2 · PictureFor curious beginners

Energy per token is the electricity used to produce one token. Measured only at the accelerator it looks smaller; a fuller count adds the host CPU and memory, machines kept idle for spare capacity, and building overhead such as cooling and power conversion.

That building overhead is captured by PUE (power usage effectiveness): total facility energy divided by the energy used by the IT equipment. A PUE close to 1 means little is lost to cooling and conversion.

Energy per token falls when the same chips serve more tokens: bigger batches, smaller or quantised models and newer, more efficient chips all help. Longer answers, reasoning steps and larger models push it up, so totals can still grow.

Like: A taxi ride's fuel: the engine, but also the air conditioning and the drive back empty.
Where it breaks: A taxi carries one fare at a time. A server can batch many users' requests together on the same chips, so the energy per token depends heavily on how busy the hardware is.

D3 · PlayFor motivated learners

Start at the chips: energy per token = accelerator power ÷ tokens per second. Then widen the boundary one step at a time: the rest of the server, the machines kept idle for busy moments, and the building's cooling and power conversion (PUE).

Change what the chips can serve and how busy the machines are. Then see how far apart a chips-only figure and an everything-counted figure end up for the same work.

Try the interactive version in the map.

Things to try

Double the tokens per second (a bigger batch on the same chips). What happens to the energy per token at every boundary?
Drop the busy share from 60% to 20%. Which boundary grows the most?
Set PUE to 1.00, the host share to 0% and the busy share to 100%. Why do the two figures now match?
Switch the answer from 500 to 10,000 tokens, about the length of a long reasoning answer. What happens to the energy per answer?

D5 · FrontierFor experts

D2 treats energy per token as one number. In practice it depends on where the boundary is drawn, what task runs, and who chooses to report.

Boundary. One provider's full-stack measurement put its median text prompt at 0.24 Wh, counting host machines, idle spare capacity and building overhead as well as accelerators. Accelerator-only figures are smaller and not comparable (Elsworth et al., 2025).

Task and model choice. General-purpose generative models were orders of magnitude more energy-hungry than task-specific models on the same tasks, even at similar size (Luccioni et al., 2023).

Rebound. Cheaper tokens invite more of them, and reasoning models and agents write long hidden chains, so total energy can rise while energy per token falls. The IEA's high-efficiency case still leaves 2035 demand only 20% below its base case (IEA, 2025).

Disclosure. The EU AI Act asks general-purpose model providers to document known or estimated energy use, but gives no shared method for the energy of serving answers (AI Act, Annex XI).

Examples as of 2026-10-30

Google's published energy figure for a median Gemini text prompt
ML.ENERGY Leaderboard
AI Energy Score (Hugging Face and partners)
PUE, the Green Grid's building-overhead metric

Sources

How it connects

Energy per token includes overhead of Power & cooling for AI data centres.
Energy per token stretches each megawatt of Grid connection & energy supply.
Batching & KV caching shares chip power across Energy per token.