bfloat16
bfloat16 is a 16-bit floating-point format used in AI training and inference, with a wider numeric range than fp16.
1 article
Last mentioned bfloat16, also written bf16, is a 16-bit floating-point format devised by Google Brain and used in AI model training and inference; the name stands for brain floating point. It has one sign bit, eight exponent bits and seven fraction bits, giving it the same exponent range as fp32. Compared with fp16, it covers a wider numeric range but has lower precision.
The format is supported by Google TPUs and many other AI accelerators and GPUs.
This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.