fp16
fp16 is the 16-bit half-precision floating-point format defined in IEEE 754, widely used in AI training and inference.
1 article
Last mentioned fp16 is the half-precision floating-point format defined in the IEEE 754 standard, consisting of 1 sign bit, 5 exponent bits and 10 significand bits. It uses half the memory per value of 32-bit fp32 and is used for AI training and inference. Compared with bf16, another 16-bit format, fp16 covers a narrower numerical range but offers more significand precision.
This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.