seniorPyTorch
How does PyTorch handle mixed precision overflow detection?
Updated May 17, 2026
Short answer
GradScaler detects overflow by checking for inf/nan gradients after scaling.
Deep explanation
During AMP training, gradients may overflow FP16 range. GradScaler monitors gradient values after unscale and reduces scaling factor if overflow is detected, ensuring numerical stability.
Unlock with a Pro subscription to view this section.
View pricingReal-world example
No real-world example available yet.
Unlock with a Pro subscription to view this section.
Upgrade to ProCommon mistakes
No common mistakes listed yet.
Unlock with a Pro subscription to view this section.
Upgrade to ProFollow-up questions
No follow-up questions available yet.
Unlock with a Pro subscription to view this section.
Upgrade to Pro