JustujuAI, layer by layer.Open in the interactive map

Cluster networking

The very fast network that lets thousands of accelerators share results every training step, so the whole cluster behaves like one computer.

Layer 3 · Cloud & Infrastructure · Build and run time · reviewed 2026-10-30

Read first: GPU clusters & orchestration.

D1 · StoryFor young children and total beginners

Predict: if one cable is slow, how many chips end up waiting?

When thousands of chips learn together, they must keep sharing notes, many times a second.

So they are joined by super fast cables, like a team passing a ball. If one pass is slow, the whole team waits.

One slow pass and the whole team waits

  1. Each chip: finishes its part
  2. Fast cables: share every chip's notes
  3. Everyone waits: for the slowest note
  4. Next step: all start together

Like: A team passing a ball: one slow pass holds everyone up.

D2 · PictureFor curious beginners

Inside one server, accelerators talk over a very fast local link. Between servers, a fabric of switches connects them, usually InfiniBand or high-speed Ethernet with RDMA, which moves data straight between memories without the CPU in the way.

After each training step, every accelerator must combine its results with all the others (a collective such as all-reduce). The step can't finish until the slowest transfer does.

That is why people say the network is the computer: adding chips only helps if the network can keep them fed, and network layout decides which jobs can run where.

Like: A team passing a ball: one slow pass holds everyone up.
Where it breaks: A team can improvise around a dropped pass. A synchronised collective can't: one faulty link or congested switch slows every chip in the job, and finding which of thousands of links is to blame is hard.

D5 · FrontierFor experts

D2 says the network must keep accelerators fed. At tens of thousands of accelerators, traffic patterns, failures and distance break the textbook fabric.

Few, huge flows. Training traffic has few distinct flows, each near line rate, so standard hash-based load balancing collides. One operator re-engineered RDMA over Ethernet for clusters of up to 32,000 GPUs (Gangidi et al., 2024).

Ethernet versus InfiniBand. An industry consortium released an Ethernet stack for AI in June 2025, pitched against vendor lock-in. Whether it displaces InfiniBand in the largest clusters is a strategic open question (Ultra Ethernet Consortium, 2025).

Failures. A 54-day run on 16,000 GPUs saw 419 unexpected interruptions, about 78% from hardware, yet kept over 90% effective training time through fast detection and restart (Grattafiori et al., 2024).

Distance. Training across sites needs far less communication. One method matched synchronous training while communicating 500 times less; whether it holds at frontier scale is open (Douillard et al., 2023).

Examples as of 2026-10-30

Sources

  1. RDMA over Ethernet for Distributed AI Training at Meta Scale (SIGCOMM 2024) (2024)
  2. The Llama 3 Herd of Models (section on training infrastructure) (2024)
  3. NVIDIA Collective Communications Library (NCCL): Overview (2025)
  4. UEC Launches Spec 1.0 Transforming Ethernet for AI and HPC at Scale (Ultra Ethernet Consortium) (2025-06-11)
  5. DiLoCo: Distributed Low-Communication Training of Language Models (Douillard et al.) (2023-11-14)

How it connects

More in Layer 3 · Cloud & Infrastructure