What is the difference between FP16 and BF16 in deep learning?

FP16 has higher precision but smaller range, while BF16 has larger range but lower precision.

FP16 uses 5-bit exponent and 10-bit mantissa, making it precise but prone to overflow. BF16 uses 8-bit exponent like FP32, making it numerically stable and preferred in large-scale training.