Why does TensorFlow training become unstable with large learning rates?

Large learning rates cause gradient updates to overshoot optimal minima.

During gradient descent, weights are updated proportionally to gradients. If the learning rate is too high, updates become too aggressive, causing divergence instead of convergence. This leads to oscillation or exploding loss. Adaptive optimizers like Adam mitigate this by adjusting step sizes per parameter.