How does mixed precision (AMP) work at hardware level?

AMP uses FP16/BF16 compute on tensor cores with FP32 master weights for stability.

Automatic Mixed Precision selectively casts operations to lower precision while keeping critical ops in FP32. GradScaler prevents underflow by scaling loss dynamically. GPUs use tensor cores for accelerated matrix multiplications.