Number formats (FP16, BF16, FP8, FP4)
How many bits each number gets inside the chip. Fewer bits mean more maths per second and more numbers per trip from memory, at some cost in precision.
Read first: GPUs & AI accelerators.
D1 · StoryFor young children and total beginners
Every number inside an AI is stored with a set number of tiny on-off switches called bits.
Fewer bits is like writing 3.1 instead of 3.14159. It is less exact, but the chip can carry and multiply far more of them at once.
Like: Writing 3.1 instead of 3.14159 so you can do more sums in the same time.
D2 · PictureFor curious beginners
A floating-point number splits its bits into a sign, an exponent (how big or small the number can be) and a mantissa (how many significant digits it keeps). FP32 uses 32 bits, FP16 and BF16 use 16, FP8 uses 8 and FP4 uses 4.
BF16 keeps the same exponent range as FP32 but fewer digits, which suits neural networks: they care more about range than precision. FP8 comes in two flavours that trade range for digits in the same way.
Halving the bits roughly doubles how many numbers fit in memory and move per second, and the chip's matrix units can do more multiplications per second in the narrower format. Very small formats such as FP4 share one scale factor across a small block of numbers so they don't run out of range.
Like: Rounding to fewer decimal places so more numbers fit on the page.
Where it breaks: Decimal rounding drops digits evenly. Floating-point formats choose how to split bits between range and precision, and FP4 also needs a shared scale per block, so the format design matters as much as the bit count.
D5 · FrontierFor experts
Training below 16 bits is still delicate: FP8 training needs careful per-tensor or per-block scaling, and FP4 training is an active research area.
Hardware support decides what is fast. A format the chip has no matrix units for gives the memory saving but not the speed-up, which is why software quantization often runs ahead of what the silicon can natively compute.
Quality loss is uneven: a few outlier values carry much of the meaning, so the same bit count can work well for one model and badly for another.