# GPUs & AI accelerators URL: https://justuju.org/concept/gpu-accelerators/ Layer 2 · Compute Hardware Chips built to do enormous numbers of simple multiplications in parallel, the core operation of neural networks. ## D1 · Story A normal computer brain (a CPU) is like one very clever person doing sums one after another. A GPU is like a stadium full of people each doing one small sum at the same time. AI needs billions of small sums, so the stadium wins. Like: One clever person versus a stadium full of helpers. ## D2 · Picture Neural networks mostly multiply large grids of numbers (matrices). GPUs and other accelerators contain thousands of simple cores and specialised matrix units that do these multiplications in parallel. They also use lower-precision number formats (16-, 8- or even 4-bit) to fit more maths per second and more numbers in memory. Each chip maker's software platform lets machine-learning frameworks use the hardware without chip-level code. Like: A CPU is a few expert chefs; a GPU is a huge team of line cooks all chopping at once. Where it breaks: Line cooks work independently. GPU cores run the same instruction on different data in lockstep, so work that can't be split evenly wastes most of the team. ## D5 · Frontier D2 explains why accelerators beat CPUs at matrix maths. It says nothing about who supplies them, what replaces them, or how fast they lose value. Vendor concentration. The leading GPU maker's filing shows concentration on both sides: two direct customers made up 22% and 14% of its fiscal 2026 revenue, and it lists cloud companies designing their own hardware as competitors (NVIDIA 10-K, 2026). Custom silicon. One cloud provider reported its fourth-generation in-house accelerator 1.2 to 1.7 times faster than a contemporary GPU on similar-sized systems, using less power. Whether in-house chips can keep pace and match a rival software ecosystem is open (Jouppi et al., 2023). Price deflation. GPU compute per dollar has doubled roughly every 2.5 years, which erodes the value of fleets bought today (Hobbhahn and Besiroglu, 2022). Memory wall. Over 20 years peak compute grew about 3.0 times every two years against 1.6 times for DRAM bandwidth, so feeding the arithmetic, not doing it, increasingly sets speed (Gholami et al., 2024). ## Examples as of 2026-10-13 - NVIDIA GPUs, programmed with CUDA - AMD GPUs, programmed with ROCm - Google TPUs - PyTorch (a framework that runs on all of these) ## Sources - Making Deep Learning Go Brrrr From First Principles (Horace He): https://horace.io/brrr_intro.html - In-Datacenter Performance Analysis of a Tensor Processing Unit: https://arxiv.org/abs/1704.04760 - NVIDIA Form 10-K, fiscal year 2026: https://www.sec.gov/Archives/edgar/data/1045810/000104581026000021/nvda-20260125.htm - TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings (Jouppi et al.): https://arxiv.org/abs/2304.01433 - Trends in GPU price-performance (Hobbhahn and Besiroglu, Epoch AI): https://epoch.ai/publications/trends-in-gpu-price-performance - AI and Memory Wall (Gholami et al.): https://arxiv.org/abs/2403.14123 --- Justuju · https://justuju.org/ · Content CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote with a link to the page.