Layer 3 · Cloud & Infrastructure

Turns racks of chips into rentable, schedulable clusters for training and serving models.

In simple words: Huge rooms of AI computers that companies rent by the hour, like renting a car instead of buying one.

reviewed 2026-10-30

Like: A utility company: you plug in and pay for what you use, without owning the power plant.
Where it breaks: Unlike electricity, AI capacity isn't interchangeable: specific chips, network layouts and regions matter, and capacity can simply be unavailable.

Key players

What it means for your business: Renting is the default: big clouds bundle AI capacity with services you already use, specialist GPU clouds can cost less but add a supplier, and long commitments trade lower prices for lock-in.

• AWS: Largest public cloud; rents GPU clusters plus its own Trainium chips. (28% of public cloud spending, Q1 2026)
• Microsoft Azure: Second-largest public cloud; rents GPU clusters for AI work. (21% of public cloud spending, Q1 2026)
• Google Cloud: Third-largest public cloud; rents both GPUs and its own TPUs. (14% of public cloud spending, Q1 2026)
• Oracle Cloud: Building large AI data-centre capacity for OpenAI under Stargate.
• CoreWeave: Specialist GPU cloud (a "neocloud") renting clusters to AI labs.
• Nebius: Amsterdam-based GPU cloud; rents dedicated capacity to Microsoft and others.
• Kubernetes: Open-source system that runs and schedules containerised services across machines.
• Slurm: Open-source job scheduler used to share large training clusters.

As of 2026-10-15. Sources: Synergy Research: cloud spending hits $129bn in Q1 2026 (DCD) (2026-04-30); Stargate advances with 4.5 GW partnership with Oracle (OpenAI) (2025-07-22); Microsoft taps Nebius in $17B AI infrastructure deal (RCP) (2025-09-12); Kubernetes overview (ongoing); Slurm workload manager overview (ongoing)

What it gives the layer above

On-demand clusters, storage and networking that teams can train and serve models on.

What it needs from the layer below

Accelerators and interconnect from Compute Hardware, housed and powered by Physical & Energy.

Concepts

• Cluster networking
  The very fast network that lets thousands of accelerators share results every training step, so the whole cluster behaves like one computer.
• GPU clusters & orchestration
  Thousands of accelerators networked and scheduled to act as one machine for training or serving.
• GPU clouds & neoclouds
  Companies that own huge GPU clusters and rent slices of them by the hour, from big general clouds to specialist GPU-only providers.
• Training vs inference infrastructure
  Training needs one giant, tightly linked cluster for weeks; serving answers needs many smaller copies close to users, sized for speed and cost per request.
• Running locally vs in the cloud
  Open-weight models can run on your own hardware at home, trading the cloud's scale and convenience for privacy, control and a fixed cost.

Walkthrough

1. Chips wired into one machine. A lab wants to train a new model. Thousands of accelerators are linked by a fast network fabric so they can share results every step.
2. Renting the cluster. The lab doesn't own that cluster. It rents it from a GPU cloud, reserving capacity for the weeks the run will take.
3. Training, then serving. Once trained, the model moves from one giant training job to many small serving copies, sized for fast, cheap answers.
4. Or run it at home. If the weights are released, a hobbyist can download a quantised version and run it on their own machine instead of renting.

Examples

• Hyperscale clouds
• Specialist GPU clouds (neoclouds)
• Cluster schedulers (Kubernetes, Slurm)
• InfiniBand and Ethernet fabrics
• Local runners (llama.cpp, Ollama)