What is entropy in Gradient Descent-based optimization?

Entropy measures uncertainty and is often used as a regularization term in optimization.

Entropy appears in optimization through maximum entropy principles and cross-entropy loss. In Gradient Descent, entropy encourages exploration and prevents overconfident predictions. It is widely used in classification models to stabilize training and improve generalization.