# Energy per token

URL: https://justuju.org/concept/energy-per-token/
Layer 1 · Physical & Energy

Each answer a model gives costs a little electricity, and how much depends on what you count: the chips, the rest of the server, idle machines and the building's cooling.

## D1 · Story

Every word the model writes uses a tiny bit of electricity.
Most goes to the chips doing the maths. Some goes to the rest of the computer, to machines waiting for work, and to cooling the building.
A bigger model, or a longer answer, uses more.
Like: A taxi ride's fuel: the engine, but also the air conditioning and the drive back empty.

## D2 · Picture

Energy per token is the electricity used to produce one token. Measured only at the accelerator it looks smaller; a fuller count adds the host CPU and memory, machines kept idle for spare capacity, and building overhead such as cooling and power conversion.
That building overhead is captured by PUE (power usage effectiveness): total facility energy divided by the energy used by the IT equipment. A PUE close to 1 means little is lost to cooling and conversion.
Energy per token falls when the same chips serve more tokens: bigger batches, smaller or quantised models and newer, more efficient chips all help. Longer answers, reasoning steps and larger models push it up, so totals can still grow.
Like: A taxi ride's fuel: the engine, but also the air conditioning and the drive back empty. Where it breaks: A taxi carries one fare at a time. A server can batch many users' requests together on the same chips, so the energy per token depends heavily on how busy the hardware is.

## D5 · Frontier

D2 treats energy per token as one number. In practice it depends on where the boundary is drawn, what task runs, and who chooses to report.
Boundary. One provider's full-stack measurement put its median text prompt at 0.24 Wh, counting host machines, idle spare capacity and building overhead as well as accelerators. Accelerator-only figures are smaller and not comparable (Elsworth et al., 2025).
Task and model choice. General-purpose generative models were orders of magnitude more energy-hungry than task-specific models on the same tasks, even at similar size (Luccioni et al., 2023).
Rebound. Cheaper tokens invite more of them, and reasoning models and agents write long hidden chains, so total energy can rise while energy per token falls. The IEA's high-efficiency case still leaves 2035 demand only 20% below its base case (IEA, 2025).
Disclosure. The EU AI Act asks general-purpose model providers to document known or estimated energy use, but gives no shared method for the energy of serving answers (AI Act, Annex XI).

## Examples as of 2026-10-30

- Google's published energy figure for a median Gemini text prompt
- ML.ENERGY Leaderboard
- AI Energy Score (Hugging Face and partners)
- PUE, the Green Grid's building-overhead metric

## Sources

- Measuring the environmental impact of delivering AI at Google Scale (Elsworth et al.): https://arxiv.org/abs/2508.15734
- Power Hungry Processing: Watts Driving the Cost of AI Deployment? (Luccioni et al.): https://arxiv.org/abs/2311.16863
- From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference (Samsi et al.): https://arxiv.org/abs/2310.03003
- Best Practices Guide for Energy-Efficient Data Center Design (US DOE FEMP / NREL), PUE definition: https://www.energy.gov/sites/default/files/2024-07/best-practice-guide-data-center-design.pdf
- IEA: Energy and AI, executive summary: https://www.iea.org/reports/energy-and-ai/executive-summary
- AI Act Annex XI: technical documentation for general-purpose AI models (EU AI Act Service Desk): https://ai-act-service-desk.ec.europa.eu/en/ai-act/annex-11


---
Justuju · https://justuju.org/ · Content CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote with a link to the page.