Chip-to-chip interconnect
The links that let several accelerators share work. Fast dedicated links let a group of chips act like one big chip; slower general-purpose links make them wait.
Read first: GPUs & AI accelerators, Memory & bandwidth (HBM).
D1 · StoryFor young children and total beginners
Big AI models are often too large for one chip, so several chips share the work.
They must keep passing notes to each other. If the note-passing is slow, the chips sit waiting instead of working.
Like: Pupils sharing one big puzzle, passing pieces across the table.
D2 · PictureFor curious beginners
A model can be split across chips in two main ways. Splitting by layers sends each chip a run of layers, and only a small bundle of results crosses between them per token. Splitting inside each layer (tensor parallelism) makes every chip do part of every layer, so they swap results many times per token.
General-purpose links such as PCIe connect a GPU to the rest of the computer and are fine for the first kind of split. Dedicated chip-to-chip links (NVLink-style, or open standards such as UALink) are much faster and can join many accelerators into one shared domain, which the second kind of split needs.
Beyond one box, network fabrics join many such groups into the clusters of the layer above.
Like: Passing notes across the table versus everyone writing on one shared whiteboard.
Where it breaks: Note-passing slows a little with each extra pupil. With chips, a slow link can make adding a second chip barely faster, or even slower, because every layer has to wait for the swap.
D5 · FrontierFor experts
The size of the fast-link domain, how many chips can talk at full speed, is now a headline feature of AI systems, and vendors compete with proprietary links against open standards.
Copper links are reaching their distance limits, so optical links and co-packaged optics are being explored. Communication can also be overlapped with computation, which blurs how much a faster link actually helps a given model.