Cost & Economics

D1 · StoryFor young children and total beginners

Guess which costs more: building the AI, or answering one question?

Making an AI costs a lot of money. Builders pay for special chips, big buildings full of computers, and lots of power.

Then each answer costs a little too. The computers must work every time someone asks a question.

So builders look for ways to do the same job with less. One trick is to save work they did before, so they need not do it twice.

Like: Building a bakery costs a lot once; every loaf after that costs a little.

D2 · PictureFor curious beginners

Costs come in two shapes. Build cost is paid once per model: chips, data, staff and the training run. Run cost is paid on every request: the computing behind each answer, usually priced per million tokens.

Training the most compute-hungry models has grown about 2.4 times more expensive each year since 2016. Chips and staff are the biggest items; energy is only 2 to 6 percent.

Run cost grows with use and with how a question is answered. Agents use about 4 times as many tokens as a chat, and multi-agent systems about 15 times.

Much optimisation work is cost work: batching many requests together, caching repeated text (on one major platform, cached input is billed at a tenth of the normal price or less), and smaller or compressed models.

Like: Building a bakery costs a lot once; every loaf after that costs a little.
Where it breaks: A loaf costs about the same every time. An AI answer can cost a hundred times more or less depending on its length, the model, and whether an agent loops many times to produce it.

D5 · FrontierFor experts

Read the capital-spending headline with care: it mixes AI with everything else the four companies build, and filings don't split it by layer, so it can't be read as AI spending per layer.

If training costs keep rising 2.4 times a year, the largest runs would pass a billion dollars by 2027, leaving frontier development to the best-funded organisations (Epoch AI, 2024).

Cost is moving from the build to every answer. The 2026 International AI Safety Report notes that recent capability gains come largely from post-training and from extra computing at the time of use, which is paid per request.

Energy is a small share of training cost but a growing share of electricity demand: the IEA projects data-centre use to more than double to around 945 TWh by 2030.

On every layer

• Layer 9 · Applications & Interfaces: Every request costs computing, so an AI app's bill grows with its use; in 2023 many AI start-ups were spending over 80% of the money they raised on compute. (Navigating the High Cost of AI Compute (Andreessen Horowitz))
• Layer 8 · Agents: Agents loop many times per task: one developer measured agents using about 4 times the tokens of a chat, and multi-agent systems about 15 times. (How we built our multi-agent research system (Anthropic))
• Layer 7 · Orchestration & Tooling: Long, repeated instructions and documents are a big part of the bill; prompt caching bills repeated input at a tenth of the normal price or less. (Prompt caching (Claude API documentation))
• Layer 6 · Inference & Optimization: Better memory management lets a server batch more requests at once: PagedAttention raised throughput 2 to 4 times at the same latency, lowering the cost of each token. (Efficient Memory Management for Large Language Model Serving with PagedAttention)
• Layer 5 · Foundation Models: The cost of training the most compute-hungry models has grown about 2.4 times a year since 2016. (The rising costs of training frontier AI models (Epoch AI))
• Layer 4 · Data: Human labels are slow and costly to collect; training from AI feedback against a written list of principles is one way to need far fewer of them. (Constitutional AI: Harmlessness from AI Feedback)
• Layer 3 · Cloud & Infrastructure: Clouds rent accelerators by the hour or in reserved blocks of days or weeks, turning a large upfront purchase into a running cost. (Amazon EC2 User Guide: Capacity Blocks for ML)
• Layer 2 · Compute Hardware: Accelerator chips and staff are the two biggest costs of training a frontier model, each reaching tens of millions of dollars. (The rising costs of training frontier AI models (Epoch AI))
• Layer 1 · Physical & Energy: Electricity is a small part of what a frontier training run costs, about 2 to 6 percent; chips and staff cost far more. (The rising costs of training frontier AI models (Epoch AI))

Sources

1. The rising costs of training frontier AI models (Epoch AI) (2024)
2. How we built our multi-agent research system (Anthropic) (2025-06-13)
3. Prompt caching (Claude API documentation) (2026)
4. International AI Safety Report 2026 (2026-02-03)
5. IEA: Energy and AI, executive summary (2025)
6. Navigating the High Cost of AI Compute (Andreessen Horowitz) (2023-04-27)