Layer 3 · Cloud & Infrastructure

Turns racks of chips into rentable, schedulable clusters for training and serving models.

In simple words: Huge rooms of AI computers that companies rent by the hour, like renting a car instead of buying one.

Draft, awaiting expert review, reviewed 2026-10-02

Like: A utility company: you plug in and pay for what you use, without owning the power plant.
Where it breaks: Unlike electricity, AI capacity isn't interchangeable: specific chips, network layouts and regions matter, and capacity can simply be unavailable.

What it gives the layer above

On-demand clusters, storage and networking that teams can train and serve models on.

What it needs from the layer below

Accelerators and interconnect from Compute Hardware, housed and powered by Physical & Energy.

Concepts

• Cluster networking
  The very fast network that lets thousands of accelerators share results every training step, so the whole cluster behaves like one computer.
• GPU clusters & orchestration
  Thousands of accelerators networked and scheduled to act as one machine for training or serving.
• GPU clouds & neoclouds
  Companies that own huge GPU clusters and rent slices of them by the hour, from big general clouds to specialist GPU-only providers.
• Training vs inference infrastructure
  Training needs one giant, tightly linked cluster for weeks; serving answers needs many smaller copies close to users, sized for speed and cost per request.
• Running locally vs in the cloud
  Open-weight models can run on your own hardware at home, trading the cloud's scale and convenience for privacy, control and a fixed cost.

Walkthrough

1. Chips wired into one machine. A lab wants to train a new model. Thousands of accelerators are linked by a fast network fabric so they can share results every step.
2. Renting the cluster. The lab doesn't own that cluster. It rents it from a GPU cloud, reserving capacity for the weeks the run will take.
3. Training, then serving. Once trained, the model moves from one giant training job to many small serving copies, sized for fast, cheap answers.
4. Or run it at home. If the weights are released, a hobbyist can download a quantised version and run it on their own machine instead of renting.

Examples

• Hyperscale clouds
• Specialist GPU clouds (neoclouds)
• Cluster schedulers (Kubernetes, Slurm)
• InfiniBand and Ethernet fabrics
• Local runners (llama.cpp, Ollama)